[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:22 dist-test-slave-7c35 krb5kdc[31082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:22 dist-test-slave-7c35 krb5kdc[31082](info): set up 2 sockets
Aug 11 05:08:22 dist-test-slave-7c35 krb5kdc[31082](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 05:08:23.278527 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:23 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:23.401932 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46659
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:46659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:23.513507 31099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:23.513779 31099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:23.513849 31099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:23.517838 31099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:23.517921 31099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:23.517957 31099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:23.517997 31099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:23.518026 31099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:23.523160 31099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:46659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:23.524168 31099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:23.525142 31099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:23.533869 31108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:23.533869 31107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:23.533905 31099 server_base.cc:1034] running on GCE node
W20260811 05:08:23.533885 31110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:23.534789 31099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:23.535892 31099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:23.537051 31099 hybrid_clock.cc:648] HybridClock initialized: now 1786424903537042 us; error 40 us; skew 500 ppm
Aug 11 05:08:23 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:23.539884 31099 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:23.540794 31099 webserver.cc:460] Webserver started at http://127.30.88.254:37193/ using document root <none> and password file <none>
I20260811 05:08:23.541343 31099 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:23.541404 31099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:23.541651 31099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:23.543766 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "3319f5c2cf8342228386401ad137a4fd"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "6c705f5a2191177947912dcfd5fbbd4b"
server_key_iv: "2e9a66197ad729514635a4fd14dab4de"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.544354 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "3319f5c2cf8342228386401ad137a4fd"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "6c705f5a2191177947912dcfd5fbbd4b"
server_key_iv: "2e9a66197ad729514635a4fd14dab4de"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.548115 31099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:23.550382 31119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:23.551627 31099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:23.551754 31099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "3319f5c2cf8342228386401ad137a4fd"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "6c705f5a2191177947912dcfd5fbbd4b"
server_key_iv: "2e9a66197ad729514635a4fd14dab4de"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.551853 31099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:23.591482 31099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:23.594506 31099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:23.594713 31099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:23.602191 31099 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:46659
I20260811 05:08:23.602221 31183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:46659 every 8 connection(s)
I20260811 05:08:23.603164 31099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:23.605933 31184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:23.608592 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31099
I20260811 05:08:23.608742 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:23.610029 31075 external_mini_cluster.cc:1428] Setting key 465a75700bbb3d536dbb07e5ffd19761
I20260811 05:08:23.611660 31184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd: Bootstrap starting.
I20260811 05:08:23.614516 31184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:23.615527 31184 log.cc:824] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:23.618275 31184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd: No bootstrap required, opened a new log
I20260811 05:08:23.623799 31184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } }
I20260811 05:08:23.624100 31184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:23.624181 31184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3319f5c2cf8342228386401ad137a4fd, State: Initialized, Role: FOLLOWER
I20260811 05:08:23.624610 31184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } }
I20260811 05:08:23.624730 31184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:23.624816 31184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:23.624912 31184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:23.626092 31184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } }
I20260811 05:08:23.626471 31184 leader_election.cc:302] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3319f5c2cf8342228386401ad137a4fd; no voters:
I20260811 05:08:23.626801 31184 leader_election.cc:288] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:23.626972 31189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:23.627208 31189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [term 1 LEADER]: Becoming Leader. State: Replica: 3319f5c2cf8342228386401ad137a4fd, State: Running, Role: LEADER
I20260811 05:08:23.627539 31189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } }
I20260811 05:08:23.628077 31184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:23.628811 31191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3319f5c2cf8342228386401ad137a4fd. Latest consensus state: current_term: 1 leader_uuid: "3319f5c2cf8342228386401ad137a4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } } }
I20260811 05:08:23.628962 31191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:23.629304 31190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3319f5c2cf8342228386401ad137a4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3319f5c2cf8342228386401ad137a4fd" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46659 } } }
I20260811 05:08:23.629398 31190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:23.629469 31199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:23.632817 31199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:08:23 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:23.638716 31199 catalog_manager.cc:1347] Generated new cluster ID: 22c83db38622414186c8b54a069e7f27
I20260811 05:08:23.638777 31199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:23.642619 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:23.613283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.30.88.254:46659)
0811 05:08:23.613733 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:23.613789 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:23.615230 (+ 1441us) server_negotiation.cc:407] Connection header received
0811 05:08:23.621161 (+ 5931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:23.621187 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:23.621582 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:23.621948 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:23.625133 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.626660 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.628434 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.629284 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.629489 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:23.635130 (+ 5641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:23.635140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:23.635158 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:23.635193 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:23.637468 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:23.638126 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:23.638133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:23.638138 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:23.638282 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:23.638739 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:23.638742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:23.638744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:23.639336 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:23.639648 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:23.641389 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0811 05:08:23.641680 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":162,"threads_started":1}
I20260811 05:08:23.666010 31199 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:23.667310 31199 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:23.677205 31199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3319f5c2cf8342228386401ad137a4fd: Generated new TSK 0
I20260811 05:08:23.678023 31199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:23.709359 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:23.820132 31212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:23.820358 31212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:23.820399 31212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:23.824386 31212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:23.824440 31212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:23.824553 31212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:23.829917 31212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:23.830957 31212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:23.831944 31212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:23.841502 31220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:23.841572 31212 server_base.cc:1034] running on GCE node
W20260811 05:08:23.841521 31218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:23.841523 31217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:23.842126 31212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:23.842733 31212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:23.843921 31212 hybrid_clock.cc:648] HybridClock initialized: now 1786424903843906 us; error 26 us; skew 500 ppm
Aug 11 05:08:23 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:23.846984 31212 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:23.847757 31212 webserver.cc:460] Webserver started at http://127.30.88.193:37183/ using document root <none> and password file <none>
I20260811 05:08:23.848255 31212 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:23.848325 31212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:23.848491 31212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:23.850443 31212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "e298e3fad4c94405a4df5ee1e8b92fd4"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "68255772b6bbd696778ac78f2d4af651"
server_key_iv: "65740658963ba1269625b117e0f24300"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.851027 31212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "e298e3fad4c94405a4df5ee1e8b92fd4"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "68255772b6bbd696778ac78f2d4af651"
server_key_iv: "65740658963ba1269625b117e0f24300"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.854538 31212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:23.857074 31227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:23.858253 31212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:23.858386 31212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "e298e3fad4c94405a4df5ee1e8b92fd4"
format_stamp: "Formatted at 2026-08-11 05:08:23 on dist-test-slave-7c35"
server_key: "68255772b6bbd696778ac78f2d4af651"
server_key_iv: "65740658963ba1269625b117e0f24300"
server_key_version: "encryptionkey@0"
I20260811 05:08:23.858497 31212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:23.869438 31212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:23.872701 31212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:23.872900 31212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:23.873459 31212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:23.874279 31212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:23.874351 31212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:23.874419 31212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:23.874475 31212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:23.883704 31212 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:34135
I20260811 05:08:23.883770 31367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:34135 every 8 connection(s)
I20260811 05:08:23.884586 31212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:23.885385 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31212
I20260811 05:08:23.885504 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:23.885774 31075 external_mini_cluster.cc:1428] Setting key 420f7d589c91fcbc5da0eda50760dc7b
Aug 11 05:08:23 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:08:23.898602 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:23.886598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:33751 (local address 127.30.88.254:46659)
0811 05:08:23.888038 (+ 1440us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:23.888042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:23.888065 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:08:23.889019 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:23.889022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:23.889086 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:23.889161 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:23.890301 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.890832 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.892123 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.892358 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.892468 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:23.894886 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:23.894890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:23.894893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:23.894925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:23.896377 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:23.896991 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:23.896995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:23.896997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:23.897086 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:23.897426 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:23.897430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:23.897432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:23.897613 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:23.897815 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:23.898303 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 05:08:23.898431 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1328}
I20260811 05:08:23.899384 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:23.887085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46659 (local address 127.30.88.193:33751)
0811 05:08:23.887561 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:23.887613 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:23.888814 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:23.889322 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:23.889329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:23.889687 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:23.890148 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:23.890161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.891022 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.891025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:23.891986 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:23.891996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:23.892506 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:23.892511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:23.892693 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:23.893287 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:23.893303 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:23.894700 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:23.896533 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:23.896539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:23.896562 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:23.896877 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:23.897185 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:23.897188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:23.897190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:23.897325 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:23.897786 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:23.897792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:23.898033 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 05:08:23.898223 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 05:08:23.898526 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":127,"threads_started":1}
I20260811 05:08:23.900678 31368 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46659
I20260811 05:08:23.900916 31368 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:23.901515 31368 heartbeater.cc:511] Master 127.30.88.254:46659 requested a full tablet report, sending...
I20260811 05:08:23.903177 31133 ts_manager.cc:194] Registered new tserver with Master: e298e3fad4c94405a4df5ee1e8b92fd4 (127.30.88.193:34135)
I20260811 05:08:23.904299 31133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:33751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:23.913781 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:24.025700 31374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:24.025993 31374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:24.026075 31374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:24.030333 31374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:24.030419 31374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:24.030516 31374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:24.035784 31374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:24.036816 31374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:24.037853 31374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:24.047974 31383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:24.048032 31379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:24.048036 31380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:24.048152 31374 server_base.cc:1034] running on GCE node
I20260811 05:08:24.048705 31374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:24.049350 31374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:24.050551 31374 hybrid_clock.cc:648] HybridClock initialized: now 1786424904050534 us; error 27 us; skew 500 ppm
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:24.053403 31374 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:24.054307 31374 webserver.cc:460] Webserver started at http://127.30.88.194:35293/ using document root <none> and password file <none>
I20260811 05:08:24.054970 31374 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:24.055076 31374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:24.055375 31374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:24.057488 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "9e18a7dd4cd64c6ea2d0f7d9e83cef35"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "e9fca7dea3878c27e35ac604fed8d146"
server_key_iv: "23575954d9114c024513cfe873f7c3bd"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.058094 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "9e18a7dd4cd64c6ea2d0f7d9e83cef35"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "e9fca7dea3878c27e35ac604fed8d146"
server_key_iv: "23575954d9114c024513cfe873f7c3bd"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.061882 31374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260811 05:08:24.064370 31393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.065510 31374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:24.065660 31374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "9e18a7dd4cd64c6ea2d0f7d9e83cef35"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "e9fca7dea3878c27e35ac604fed8d146"
server_key_iv: "23575954d9114c024513cfe873f7c3bd"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.065775 31374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:24.088181 31374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:24.090962 31374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:24.091154 31374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:24.091714 31374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:24.092530 31374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:24.092602 31374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.092675 31374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:24.092720 31374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.102126 31374 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:46883
I20260811 05:08:24.102159 31529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:46883 every 8 connection(s)
I20260811 05:08:24.103120 31374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:24.109671 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31374
I20260811 05:08:24.109768 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:24.110040 31075 external_mini_cluster.cc:1428] Setting key c3d68df489ada60dc970ec2ed4f2fb6c
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:24.117501 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.104928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:47585 (local address 127.30.88.254:46659)
0811 05:08:24.105073 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:24.105077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:24.105860 (+ 783us) server_negotiation.cc:407] Connection header received
0811 05:08:24.107283 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:24.107287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:24.107452 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:24.107611 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:24.108755 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.109290 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.110592 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.110907 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.111004 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.113720 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:24.113726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:24.113734 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:24.113758 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:24.115294 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.115822 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.115826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.115827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.115910 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.116299 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.116302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.116304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.116597 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:24.116693 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:24.117174 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 05:08:24.117321 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:08:24.118242 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.105235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46659 (local address 127.30.88.194:47585)
0811 05:08:24.105701 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:24.105742 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:24.107057 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:24.107791 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:24.107797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:24.108156 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:24.108607 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:24.108621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.109432 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.109440 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:24.110458 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:24.110468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.111063 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.111066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:24.111209 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.111862 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:24.111885 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:24.113553 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:24.115426 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:24.115432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:24.115445 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:24.115715 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:24.116024 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:24.116027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:24.116030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:24.116175 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:24.116715 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:24.116721 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:24.116938 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 05:08:24.117127 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 05:08:24.117464 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
I20260811 05:08:24.119417 31530 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46659
I20260811 05:08:24.119678 31530 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:24.120151 31530 heartbeater.cc:511] Master 127.30.88.254:46659 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:08:24.121254 31133 ts_manager.cc:194] Registered new tserver with Master: 9e18a7dd4cd64c6ea2d0f7d9e83cef35 (127.30.88.194:46883)
I20260811 05:08:24.121886 31133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:47585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:24.137786 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:24.251405 31535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:24.251628 31535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:24.251669 31535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:24.255929 31535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:24.255990 31535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:24.256050 31535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:24.261041 31535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:24.261983 31535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:24.263034 31535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:24.272612 31543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:24.272687 31540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:24.272624 31541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:24.272874 31535 server_base.cc:1034] running on GCE node
I20260811 05:08:24.273451 31535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:24.274070 31535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:24.275264 31535 hybrid_clock.cc:648] HybridClock initialized: now 1786424904275246 us; error 32 us; skew 500 ppm
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:24.277933 31535 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:24.278786 31535 webserver.cc:460] Webserver started at http://127.30.88.195:44251/ using document root <none> and password file <none>
I20260811 05:08:24.279376 31535 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:24.279467 31535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:24.279717 31535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:24.281669 31535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "e63252f3ac264fac93f196297a8c5956"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "7ee93a1c4c87c06ecc123e8992739a29"
server_key_iv: "dde70b30fa91eb97ee3615918698e10b"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.282227 31535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "e63252f3ac264fac93f196297a8c5956"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "7ee93a1c4c87c06ecc123e8992739a29"
server_key_iv: "dde70b30fa91eb97ee3615918698e10b"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.285789 31535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:24.288214 31551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.289342 31535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:24.289474 31535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "e63252f3ac264fac93f196297a8c5956"
format_stamp: "Formatted at 2026-08-11 05:08:24 on dist-test-slave-7c35"
server_key: "7ee93a1c4c87c06ecc123e8992739a29"
server_key_iv: "dde70b30fa91eb97ee3615918698e10b"
server_key_version: "encryptionkey@0"
I20260811 05:08:24.289582 31535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:24.304199 31535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:24.306771 31535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:24.306999 31535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:24.307546 31535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:24.308352 31535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:24.308424 31535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.308502 31535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:24.308545 31535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:24.317726 31535 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39035
I20260811 05:08:24.317755 31690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39035 every 8 connection(s)
I20260811 05:08:24.318568 31535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:24.323865 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31535
I20260811 05:08:24.323990 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:24.324283 31075 external_mini_cluster.cc:1428] Setting key 54c3103666adea44e63814a3b859b003
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:24.332285 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.320377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:35915 (local address 127.30.88.254:46659)
0811 05:08:24.320537 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:24.320542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:24.321361 (+ 819us) server_negotiation.cc:407] Connection header received
0811 05:08:24.322555 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:24.322558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:24.322604 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:24.322678 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:24.324028 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.324692 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.325905 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.326141 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.326265 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.328929 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:24.328932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:24.328934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:24.328959 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:24.330391 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.330951 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.330955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.330956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.331029 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.331367 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.331369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.331370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.331547 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:24.331628 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:24.332067 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 05:08:24.332164 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:08:24.333032 31695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.320768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46659 (local address 127.30.88.195:35915)
0811 05:08:24.321209 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:24.321250 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:24.322359 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:24.322855 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:24.322862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:24.323249 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:24.323805 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:24.323827 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.324860 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.324864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:24.325751 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:24.325762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.326303 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.326308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:24.326505 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.327126 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:24.327144 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:24.328690 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:24.330507 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:24.330511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:24.330522 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:24.330796 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:24.331137 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:24.331139 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:24.331140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:24.331273 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:24.331643 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:24.331648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:24.331855 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 05:08:24.332001 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 05:08:24.332240 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":121,"threads_started":1}
I20260811 05:08:24.334122 31691 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46659
I20260811 05:08:24.334353 31691 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:24.334824 31691 heartbeater.cc:511] Master 127.30.88.254:46659 requested a full tablet report, sending...
I20260811 05:08:24.335865 31133 ts_manager.cc:194] Registered new tserver with Master: e63252f3ac264fac93f196297a8c5956 (127.30.88.195:39035)
I20260811 05:08:24.336500 31133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:35915
I20260811 05:08:24.338430 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:24.365568 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.355895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.30.88.193:34135)
0811 05:08:24.356222 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:24.356235 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:24.356263 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:08:24.356432 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:24.356438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:24.356705 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:24.356868 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:24.357567 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.358359 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.359252 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.359624 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.359762 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.362154 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:24.362163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:24.362174 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:24.362198 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:24.363938 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.364319 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.364324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.364325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.364400 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.364625 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.364628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.364629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.364880 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:24.365048 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:24.365201 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:08:24.365360 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
W20260811 05:08:24.366519 31259 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:24.392403 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:24.384552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.30.88.193:34135)
0811 05:08:24.384715 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:24.384719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:24.384781 (+ 62us) server_negotiation.cc:407] Connection header received
0811 05:08:24.384889 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:24.384891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:24.384930 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:24.384999 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:24.385651 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.386167 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.387021 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:24.387245 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:24.387323 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:24.389505 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:24.389508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:24.389510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:24.389531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:24.390750 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.391160 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.391163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.391164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.391236 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:24.391467 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:24.391469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:24.391470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:24.391648 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:24.391799 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:24.392093 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 05:08:24.392227 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:08:24.393471 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31212
I20260811 05:08:24.401579 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31374
I20260811 05:08:24.413002 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31535
I20260811 05:08:24.421474 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31099
2026-08-11T05:08:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2278 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31708](info): set up 2 sockets
Aug 11 05:08:24 dist-test-slave-7c35 krb5kdc[31708](info): commencing operation
krb5kdc: starting...
W20260811 05:08:25.530777 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:25 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:25.657295 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:44069
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:25.768026 31730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:25.768327 31730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:25.768395 31730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:25.772342 31730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:25.772428 31730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:25.772466 31730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:25.772503 31730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:25.772538 31730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:25.777653 31730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:44069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:25.778766 31730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:25.779788 31730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:25.787969 31739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:25.788153 31730 server_base.cc:1034] running on GCE node
W20260811 05:08:25.788182 31736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:25.788017 31737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:25.788806 31730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:25.789872 31730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:25.791036 31730 hybrid_clock.cc:648] HybridClock initialized: now 1786424905791021 us; error 26 us; skew 500 ppm
Aug 11 05:08:25 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:25.797317 31730 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:25.798204 31730 webserver.cc:460] Webserver started at http://127.30.88.254:46357/ using document root <none> and password file <none>
I20260811 05:08:25.798743 31730 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:25.798832 31730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:25.799065 31730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:25.801010 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "dab4c8579044495ca2bfe45ec3abd9ab"
format_stamp: "Formatted at 2026-08-11 05:08:25 on dist-test-slave-7c35"
server_key: "5046795f9c9041de1ae655e436eec9f1"
server_key_iv: "c7a3d829f63154c4a838b560e56aad51"
server_key_version: "encryptionkey@0"
I20260811 05:08:25.801582 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "dab4c8579044495ca2bfe45ec3abd9ab"
format_stamp: "Formatted at 2026-08-11 05:08:25 on dist-test-slave-7c35"
server_key: "5046795f9c9041de1ae655e436eec9f1"
server_key_iv: "c7a3d829f63154c4a838b560e56aad51"
server_key_version: "encryptionkey@0"
I20260811 05:08:25.805198 31730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 05:08:25.807435 31746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:25.808583 31730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:08:25.808717 31730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "dab4c8579044495ca2bfe45ec3abd9ab"
format_stamp: "Formatted at 2026-08-11 05:08:25 on dist-test-slave-7c35"
server_key: "5046795f9c9041de1ae655e436eec9f1"
server_key_iv: "c7a3d829f63154c4a838b560e56aad51"
server_key_version: "encryptionkey@0"
I20260811 05:08:25.808821 31730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:25.833724 31730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:25.836607 31730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:25.836818 31730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:25.844257 31730 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:44069
I20260811 05:08:25.844283 31803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:44069 every 8 connection(s)
I20260811 05:08:25.845198 31730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:25.847878 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:25.853104 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31730
I20260811 05:08:25.853224 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:25.853499 31075 external_mini_cluster.cc:1428] Setting key 7a6c5375b6ba6bf430cc7fce1cc4e3db
I20260811 05:08:25.853536 31806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab: Bootstrap starting.
I20260811 05:08:25.855978 31806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:25.856694 31806 log.cc:824] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:25.858511 31806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab: No bootstrap required, opened a new log
Aug 11 05:08:25 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:25.862411 31806 raft_consensus.cc:348] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } }
I20260811 05:08:25.862651 31806 raft_consensus.cc:374] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:25.862730 31806 raft_consensus.cc:729] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dab4c8579044495ca2bfe45ec3abd9ab, State: Initialized, Role: FOLLOWER
I20260811 05:08:25.863270 31806 consensus_queue.cc:260] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } }
I20260811 05:08:25.863469 31806 raft_consensus.cc:388] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:25.863565 31806 raft_consensus.cc:482] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:25.863672 31806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:25.864513 31806 raft_consensus.cc:504] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } }
I20260811 05:08:25.864848 31806 leader_election.cc:302] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dab4c8579044495ca2bfe45ec3abd9ab; no voters:
I20260811 05:08:25.865161 31806 leader_election.cc:288] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:25.865260 31812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:25.865490 31812 raft_consensus.cc:686] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [term 1 LEADER]: Becoming Leader. State: Replica: dab4c8579044495ca2bfe45ec3abd9ab, State: Running, Role: LEADER
I20260811 05:08:25.865804 31812 consensus_queue.cc:237] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } }
I20260811 05:08:25.866432 31806 sys_catalog.cc:564] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:25.867467 31813 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } } }
I20260811 05:08:25.867601 31813 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:25.867933 31814 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader dab4c8579044495ca2bfe45ec3abd9ab. Latest consensus state: current_term: 1 leader_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab4c8579044495ca2bfe45ec3abd9ab" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44069 } } }
I20260811 05:08:25.867969 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:25.854877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.30.88.254:44069)
0811 05:08:25.855291 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:25.855306 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:25.855340 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:08:25.855948 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:25.855964 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:25.856323 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:25.856654 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:25.857452 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:25.858409 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:25.859362 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:25.859834 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:25.860007 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:25.862671 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:25.862680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:25.862696 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:25.862728 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:25.864569 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:25.865074 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:25.865082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:25.865093 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:25.865246 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:25.865600 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:25.865604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:25.865606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:25.866093 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:25.866301 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:25.866588 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 05:08:25.866921 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260811 05:08:25.868026 31814 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:25.868010 31822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:25.871462 31822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:25.876924 31822 catalog_manager.cc:1347] Generated new cluster ID: 1e5e989e8a3e4af58e8c5997e8f19a5a
I20260811 05:08:25.877003 31822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:25.884871 31822 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:25.885986 31822 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:25.894050 31822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dab4c8579044495ca2bfe45ec3abd9ab: Generated new TSK 0
I20260811 05:08:25.894713 31822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:25.930189 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:26.068392 31834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:26.068611 31834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:26.068651 31834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:26.072528 31834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:26.072573 31834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:26.072628 31834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:26.077500 31834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:26.078378 31834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:26.079278 31834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:26.088343 31844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.088361 31840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.088420 31841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:26.088762 31834 server_base.cc:1034] running on GCE node
I20260811 05:08:26.089191 31834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:26.089757 31834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:26.090921 31834 hybrid_clock.cc:648] HybridClock initialized: now 1786424906090891 us; error 40 us; skew 500 ppm
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:26.093590 31834 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:26.094345 31834 webserver.cc:460] Webserver started at http://127.30.88.193:34381/ using document root <none> and password file <none>
I20260811 05:08:26.094920 31834 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:26.095009 31834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:26.095213 31834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:26.097230 31834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "65803ccba6d042468f4204318119c867"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "adddf732eb79811c6dc1e406090574b1"
server_key_iv: "2d4686bd822f9d542c900890497f78e6"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.097783 31834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "65803ccba6d042468f4204318119c867"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "adddf732eb79811c6dc1e406090574b1"
server_key_iv: "2d4686bd822f9d542c900890497f78e6"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.101274 31834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:26.103557 31855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.104631 31834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:26.104761 31834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "65803ccba6d042468f4204318119c867"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "adddf732eb79811c6dc1e406090574b1"
server_key_iv: "2d4686bd822f9d542c900890497f78e6"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.104863 31834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:26.117127 31834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:26.120782 31834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:26.120980 31834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:26.121536 31834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:26.122341 31834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:26.122412 31834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.122478 31834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:26.122536 31834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.131088 31834 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:35827
I20260811 05:08:26.131131 31988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:35827 every 8 connection(s)
I20260811 05:08:26.131913 31834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:26.136451 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31834
I20260811 05:08:26.136545 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:26.136801 31075 external_mini_cluster.cc:1428] Setting key 87f7dd18c153ab3647ebce2c232f5e9b
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:26.145726 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.133593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:45681 (local address 127.30.88.254:44069)
0811 05:08:26.133761 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.133767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.134539 (+ 772us) server_negotiation.cc:407] Connection header received
0811 05:08:26.135397 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.135400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.135449 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.135525 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.136717 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.137361 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.138872 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.139297 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.139430 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.141901 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.141905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.141907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.141930 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.143461 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.144080 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.144084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.144086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.144170 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.144558 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.144561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.144563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.144747 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.144855 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.145395 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.145545 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:08:26.146510 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.133951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:44069 (local address 127.30.88.193:45681)
0811 05:08:26.134386 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.134425 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.135197 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.135680 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.135686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.136009 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:26.136542 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.136564 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.137536 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.137540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.138707 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.138718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.139445 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.139448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.139598 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.140191 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:26.140207 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:26.141718 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:26.143607 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.143613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.143626 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.143961 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.144289 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.144292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.144294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.144453 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.144857 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:26.144864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:26.145162 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.145352 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.145680 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:26.147802 31989 heartbeater.cc:347] Connected to a master server at 127.30.88.254:44069
I20260811 05:08:26.148044 31989 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:26.148553 31989 heartbeater.cc:511] Master 127.30.88.254:44069 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:08:26.150158 31765 ts_manager.cc:194] Registered new tserver with Master: 65803ccba6d042468f4204318119c867 (127.30.88.193:35827)
I20260811 05:08:26.151822 31765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:45681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:26.165668 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:26.276726 31994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:26.276991 31994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:26.277070 31994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:26.281013 31994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:26.281093 31994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:26.281184 31994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:26.286208 31994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:26.287248 31994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:26.288169 31994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:08:26.297602 31994 server_base.cc:1034] running on GCE node
W20260811 05:08:26.297537 32000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.297570 31999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.297554 32002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:26.298136 31994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:26.298748 31994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:26.299937 31994 hybrid_clock.cc:648] HybridClock initialized: now 1786424906299920 us; error 33 us; skew 500 ppm
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:26.302595 31994 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:26.303490 31994 webserver.cc:460] Webserver started at http://127.30.88.194:46767/ using document root <none> and password file <none>
I20260811 05:08:26.304023 31994 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:26.304116 31994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:26.304353 31994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:26.306231 31994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "3b53f1dad10c413fbd3b531cd48a87f0"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "d47146ff5ca6d0bfa2468127563f6942"
server_key_iv: "4f4a2ae0452383d902503d6c2c7a3941"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.306835 31994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "3b53f1dad10c413fbd3b531cd48a87f0"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "d47146ff5ca6d0bfa2468127563f6942"
server_key_iv: "4f4a2ae0452383d902503d6c2c7a3941"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.310393 31994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:08:26.312836 32011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.314085 31994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:26.314226 31994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "3b53f1dad10c413fbd3b531cd48a87f0"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "d47146ff5ca6d0bfa2468127563f6942"
server_key_iv: "4f4a2ae0452383d902503d6c2c7a3941"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.314337 31994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:26.329021 31994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:26.331851 31994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:26.332052 31994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:26.332602 31994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:26.333405 31994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:26.333477 31994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.333546 31994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:26.333601 31994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.344024 31994 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:37019
I20260811 05:08:26.344072 32148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:37019 every 8 connection(s)
I20260811 05:08:26.344915 31994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:26.351656 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 31994
I20260811 05:08:26.351787 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:26.352079 31075 external_mini_cluster.cc:1428] Setting key fe5b6cd5768cfa95886cab0d7c154368
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:26.358470 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.346832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:48513 (local address 127.30.88.254:44069)
0811 05:08:26.347008 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.347012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.347659 (+ 647us) server_negotiation.cc:407] Connection header received
0811 05:08:26.348505 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.348508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.348553 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.348626 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.349692 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.350232 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.351423 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.351649 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.351756 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.354439 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.354445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.354469 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.354492 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.356252 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.356774 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.356779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.356782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.356893 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.357223 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.357230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.357234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.357513 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.357618 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.358096 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.358229 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 05:08:26.359112 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.347077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:44069 (local address 127.30.88.194:48513)
0811 05:08:26.347500 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.347540 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.348303 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.348785 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.348791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.349126 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:26.349561 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.349574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.350342 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.350344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.351291 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.351301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.351809 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.351816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.352012 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.352626 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:26.352641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:26.354254 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:26.356375 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.356380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.356391 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.356672 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.356990 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.356992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.356993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.357126 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.357642 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:26.357646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:26.357863 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.358014 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.358243 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260811 05:08:26.360394 32149 heartbeater.cc:347] Connected to a master server at 127.30.88.254:44069
I20260811 05:08:26.360633 32149 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:26.361089 32149 heartbeater.cc:511] Master 127.30.88.254:44069 requested a full tablet report, sending...
I20260811 05:08:26.362159 31765 ts_manager.cc:194] Registered new tserver with Master: 3b53f1dad10c413fbd3b531cd48a87f0 (127.30.88.194:37019)
I20260811 05:08:26.363034 31765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:48513
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:26.381587 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:26.492823 32155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:26.493094 32155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:26.493176 32155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:26.497352 32155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:26.497430 32155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:26.497522 32155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:26.502437 32155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:44069
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:26.503466 32155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:26.504395 32155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:26.513702 32162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.513703 32161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:26.513703 32164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:26.513868 32155 server_base.cc:1034] running on GCE node
I20260811 05:08:26.514338 32155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:26.514936 32155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:26.516114 32155 hybrid_clock.cc:648] HybridClock initialized: now 1786424906516092 us; error 31 us; skew 500 ppm
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:26.518976 32155 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:26.519716 32155 webserver.cc:460] Webserver started at http://127.30.88.195:35529/ using document root <none> and password file <none>
I20260811 05:08:26.520229 32155 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:26.520287 32155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:26.520445 32155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:26.522603 32155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "dde6e05567a941e69d526ee708433716"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "c82bc0e8c7e0cf4bdcca2903ae06ab10"
server_key_iv: "03164cb023c77f5ee2145f8df9e7040d"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.523269 32155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "dde6e05567a941e69d526ee708433716"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "c82bc0e8c7e0cf4bdcca2903ae06ab10"
server_key_iv: "03164cb023c77f5ee2145f8df9e7040d"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.527211 32155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:08:26.529865 32173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.531075 32155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:26.531193 32155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "dde6e05567a941e69d526ee708433716"
format_stamp: "Formatted at 2026-08-11 05:08:26 on dist-test-slave-7c35"
server_key: "c82bc0e8c7e0cf4bdcca2903ae06ab10"
server_key_iv: "03164cb023c77f5ee2145f8df9e7040d"
server_key_version: "encryptionkey@0"
I20260811 05:08:26.531292 32155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:26.553462 32155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:26.556974 32155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:26.557201 32155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:26.557830 32155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:26.558701 32155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:26.558753 32155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.558826 32155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:26.558897 32155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:26.569012 32155 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:40465
I20260811 05:08:26.569054 32308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:40465 every 8 connection(s)
I20260811 05:08:26.569916 32155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:26.577818 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 32155
I20260811 05:08:26.577950 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:26.578264 31075 external_mini_cluster.cc:1428] Setting key e201eac2edcae561f6e00329842c813a
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:26.583642 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.571792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:42083 (local address 127.30.88.254:44069)
0811 05:08:26.571956 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.571961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.572750 (+ 789us) server_negotiation.cc:407] Connection header received
0811 05:08:26.573690 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.573694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.573757 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.573871 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.575140 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.575726 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.576947 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.577182 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.577307 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.580241 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.580245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.580248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.580281 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.581824 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.582292 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.582294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.582296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.582370 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.582674 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.582677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.582678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.582841 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.582975 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.583398 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.583495 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:08:26.584411 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.572136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:44069 (local address 127.30.88.195:42083)
0811 05:08:26.572597 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.572640 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.573473 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.574056 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.574063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.574422 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:26.574989 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.575006 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.575879 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.575883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.576796 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.576807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.577346 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.577352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.577538 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.578202 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:26.578220 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:26.580072 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:26.581931 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.581936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.581946 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.582199 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.582453 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:26.582455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:26.582456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:26.582589 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:26.582978 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:26.582983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:26.583193 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.583361 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.583628 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
I20260811 05:08:26.585529 32309 heartbeater.cc:347] Connected to a master server at 127.30.88.254:44069
I20260811 05:08:26.585785 32309 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:26.586272 32309 heartbeater.cc:511] Master 127.30.88.254:44069 requested a full tablet report, sending...
I20260811 05:08:26.587286 31765 ts_manager.cc:194] Registered new tserver with Master: dde6e05567a941e69d526ee708433716 (127.30.88.195:40465)
I20260811 05:08:26.587863 31765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:42083
I20260811 05:08:26.592257 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:08:26.602387 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.595777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.30.88.254:44069)
0811 05:08:26.595902 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.595906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.596110 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:08:26.596199 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.596201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.596235 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.596302 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.596935 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.597408 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.598203 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.598401 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.598476 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.599753 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.599756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.599758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.599778 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.601003 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.601374 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.601376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.601377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.601444 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.601667 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.601669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.601670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.601887 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.602001 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.602194 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.602281 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:08:26.606745 31765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:26.609889 31765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:26.624653 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.618593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.30.88.193:35827)
0811 05:08:26.619446 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.619462 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.619488 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:26.619585 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.619590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.619798 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.620213 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:26.620392 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621598 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.623362 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.623986 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624112 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624258 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.624408 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.624511 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":129,"threads_started":1}
I20260811 05:08:26.624636 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.618471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35827 (local address 127.0.0.1:52162)
0811 05:08:26.618796 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.618882 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.619027 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.619985 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.619994 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.620023 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:26.620193 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.620201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621778 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.621781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.623225 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.623236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.624114 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.624221 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624257 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.624370 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.624491 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":179,"thread_start_us":76,"threads_started":1}
I20260811 05:08:26.624876 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.619028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:40465 (local address 127.0.0.1:41072)
0811 05:08:26.619279 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.619293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.619382 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.620103 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.620106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.620120 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:26.620361 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.620369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621831 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.621838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.623209 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.623221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.624319 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.624433 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624455 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.624503 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.624589 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":334,"thread_start_us":120,"threads_started":1}
I20260811 05:08:26.625159 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.619278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.30.88.195:40465)
0811 05:08:26.619681 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.619691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.619710 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:08:26.619777 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.619781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.619947 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.620505 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:26.620651 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621719 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.623354 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.624171 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624710 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624849 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.624903 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.625020 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":130,"threads_started":1}
I20260811 05:08:26.625200 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.619077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:37019 (local address 127.0.0.1:51006)
0811 05:08:26.620015 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.620030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.620120 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.620676 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.620680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.620698 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:26.620787 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.620794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621734 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.621739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.623407 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.623415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.624622 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.624710 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624934 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.625001 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.625081 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":271,"thread_start_us":105,"threads_started":1}
I20260811 05:08:26.625651 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.619519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.30.88.194:37019)
0811 05:08:26.619816 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.619827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.620221 (+ 394us) server_negotiation.cc:407] Connection header received
0811 05:08:26.620306 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.620313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.620519 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.620687 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:26.620883 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.621598 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.623524 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.624330 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.624425 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.624562 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.625410 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.625512 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":123,"threads_started":1}
I20260811 05:08:26.627856 32232 tablet_service.cc:1467] Processing CreateTablet for tablet c84342ef032540038740312f42b6ad28 (DEFAULT_TABLE table=test-table [id=d1d219b25ba54b4d9d884ded2c0a4602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:26.628036 31911 tablet_service.cc:1467] Processing CreateTablet for tablet c84342ef032540038740312f42b6ad28 (DEFAULT_TABLE table=test-table [id=d1d219b25ba54b4d9d884ded2c0a4602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:26.628834 32232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84342ef032540038740312f42b6ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:26.628958 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84342ef032540038740312f42b6ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:26.634706 32328 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Bootstrap starting.
I20260811 05:08:26.635636 32329 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Bootstrap starting.
I20260811 05:08:26.637285 32328 tablet_bootstrap.cc:654] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:26.636871 32072 tablet_service.cc:1467] Processing CreateTablet for tablet c84342ef032540038740312f42b6ad28 (DEFAULT_TABLE table=test-table [id=d1d219b25ba54b4d9d884ded2c0a4602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:26.637702 32329 tablet_bootstrap.cc:654] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:26.638089 32072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84342ef032540038740312f42b6ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:26.638528 32329 log.cc:824] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:26.638600 32328 log.cc:824] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:26.640662 32329 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: No bootstrap required, opened a new log
I20260811 05:08:26.640877 32329 ts_tablet_manager.cc:1397] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 05:08:26.641436 32328 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: No bootstrap required, opened a new log
I20260811 05:08:26.641644 32328 ts_tablet_manager.cc:1397] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 05:08:26.644856 32328 raft_consensus.cc:348] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.644915 32329 raft_consensus.cc:348] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.645115 32328 raft_consensus.cc:374] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:26.645181 32328 raft_consensus.cc:729] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65803ccba6d042468f4204318119c867, State: Initialized, Role: FOLLOWER
I20260811 05:08:26.645185 32329 raft_consensus.cc:374] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:26.645239 32329 raft_consensus.cc:729] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde6e05567a941e69d526ee708433716, State: Initialized, Role: FOLLOWER
I20260811 05:08:26.645628 32333 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Bootstrap starting.
I20260811 05:08:26.645656 32328 consensus_queue.cc:260] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.645782 32329 consensus_queue.cc:260] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.646409 31989 heartbeater.cc:503] Master 127.30.88.254:44069 was elected leader, sending a full tablet report...
I20260811 05:08:26.646701 32328 ts_tablet_manager.cc:1428] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:08:26.646713 32329 ts_tablet_manager.cc:1428] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 05:08:26.647039 32309 heartbeater.cc:503] Master 127.30.88.254:44069 was elected leader, sending a full tablet report...
I20260811 05:08:26.649243 32333 tablet_bootstrap.cc:654] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:26.650068 32333 log.cc:824] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:26.652055 32333 tablet_bootstrap.cc:492] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: No bootstrap required, opened a new log
I20260811 05:08:26.652256 32333 ts_tablet_manager.cc:1397] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 05:08:26.654960 32333 raft_consensus.cc:348] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.655184 32333 raft_consensus.cc:374] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:26.655244 32333 raft_consensus.cc:729] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b53f1dad10c413fbd3b531cd48a87f0, State: Initialized, Role: FOLLOWER
I20260811 05:08:26.655674 32333 consensus_queue.cc:260] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.656337 32149 heartbeater.cc:503] Master 127.30.88.254:44069 was elected leader, sending a full tablet report...
I20260811 05:08:26.656612 32333 ts_tablet_manager.cc:1428] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260811 05:08:26.820887 32310 tablet.cc:2369] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:26.834069 32336 raft_consensus.cc:482] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:26.834326 32336 raft_consensus.cc:504] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.835573 32336 leader_election.cc:288] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65803ccba6d042468f4204318119c867 (127.30.88.193:35827), 3b53f1dad10c413fbd3b531cd48a87f0 (127.30.88.194:37019)
I20260811 05:08:26.840473 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.835965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:52385 (local address 127.30.88.194:37019)
0811 05:08:26.836089 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.836095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.836501 (+ 406us) server_negotiation.cc:407] Connection header received
0811 05:08:26.836605 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.836607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.836649 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.836709 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:26.837002 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.837708 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839180 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.839796 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839921 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.839975 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.840265 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.840335 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:26.840472 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.835983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:35863 (local address 127.30.88.193:35827)
0811 05:08:26.836145 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.836153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.836174 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:08:26.836308 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.836313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.836374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.836445 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:26.836799 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.837588 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839180 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.839796 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839921 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.839975 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.840265 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.840335 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:08:26.840489 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.835863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35827 (local address 127.30.88.195:35863)
0811 05:08:26.836031 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.836048 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.836162 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.836486 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.836490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.836527 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:26.836663 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.836669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.837703 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.837705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.839054 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.839065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.839927 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.840096 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.840147 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.840232 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.840321 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":24,"spinlock_wait_cycles":35584}
I20260811 05:08:26.840497 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.835863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:37019 (local address 127.30.88.195:52385)
0811 05:08:26.836378 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:26.836388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:26.836508 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:26.836745 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:26.836751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:26.836773 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:26.836889 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.836895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.837805 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.837808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.839059 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:26.839065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.839927 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.839931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:26.840091 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.840160 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 05:08:26.840231 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:08:26.840321 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":240,"thread_start_us":151,"threads_started":1}
I20260811 05:08:26.841293 31934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84342ef032540038740312f42b6ad28" candidate_uuid: "dde6e05567a941e69d526ee708433716" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65803ccba6d042468f4204318119c867" is_pre_election: true
I20260811 05:08:26.841416 32097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84342ef032540038740312f42b6ad28" candidate_uuid: "dde6e05567a941e69d526ee708433716" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" is_pre_election: true
I20260811 05:08:26.841624 31934 raft_consensus.cc:2454] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde6e05567a941e69d526ee708433716 in term 0.
I20260811 05:08:26.841747 32097 raft_consensus.cc:2454] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde6e05567a941e69d526ee708433716 in term 0.
I20260811 05:08:26.842168 32174 leader_election.cc:302] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65803ccba6d042468f4204318119c867, dde6e05567a941e69d526ee708433716; no voters:
I20260811 05:08:26.842445 32336 raft_consensus.cc:2781] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:26.842535 32336 raft_consensus.cc:482] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:26.842579 32336 raft_consensus.cc:3037] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:26.843734 32336 raft_consensus.cc:504] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.844106 32336 leader_election.cc:288] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [CANDIDATE]: Term 1 election: Requested vote from peers 65803ccba6d042468f4204318119c867 (127.30.88.193:35827), 3b53f1dad10c413fbd3b531cd48a87f0 (127.30.88.194:37019)
I20260811 05:08:26.844414 31934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84342ef032540038740312f42b6ad28" candidate_uuid: "dde6e05567a941e69d526ee708433716" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65803ccba6d042468f4204318119c867"
I20260811 05:08:26.844482 32097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84342ef032540038740312f42b6ad28" candidate_uuid: "dde6e05567a941e69d526ee708433716" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b53f1dad10c413fbd3b531cd48a87f0"
I20260811 05:08:26.844547 31934 raft_consensus.cc:3037] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:26.844596 32097 raft_consensus.cc:3037] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:26.845891 31934 raft_consensus.cc:2454] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde6e05567a941e69d526ee708433716 in term 1.
I20260811 05:08:26.845891 32097 raft_consensus.cc:2454] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde6e05567a941e69d526ee708433716 in term 1.
W20260811 05:08:26.846017 32150 tablet.cc:2369] T c84342ef032540038740312f42b6ad28 P 3b53f1dad10c413fbd3b531cd48a87f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:26.846192 32174 leader_election.cc:302] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65803ccba6d042468f4204318119c867, dde6e05567a941e69d526ee708433716; no voters:
I20260811 05:08:26.846356 32336 raft_consensus.cc:2781] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:26.846546 32336 raft_consensus.cc:686] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [term 1 LEADER]: Becoming Leader. State: Replica: dde6e05567a941e69d526ee708433716, State: Running, Role: LEADER
I20260811 05:08:26.846925 32336 consensus_queue.cc:237] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } }
I20260811 05:08:26.850312 31764 catalog_manager.cc:5495] T c84342ef032540038740312f42b6ad28 P dde6e05567a941e69d526ee708433716 reported cstate change: term changed from 0 to 1, leader changed from <none> to dde6e05567a941e69d526ee708433716 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "dde6e05567a941e69d526ee708433716" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65803ccba6d042468f4204318119c867" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b53f1dad10c413fbd3b531cd48a87f0" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dde6e05567a941e69d526ee708433716" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40465 } health_report { overall_health: HEALTHY } } }
W20260811 05:08:26.883142 31990 tablet.cc:2369] T c84342ef032540038740312f42b6ad28 P 65803ccba6d042468f4204318119c867: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:26.916121 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.906945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.30.88.193:35827)
0811 05:08:26.907102 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.907107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.907301 (+ 194us) server_negotiation.cc:407] Connection header received
0811 05:08:26.907456 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.907459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.907517 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.907601 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.908286 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.908776 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.909654 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.909857 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.909956 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.912671 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.912681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.912697 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.912719 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.914257 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.914722 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.914728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.914729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.914814 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.915183 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.915186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.915188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.915446 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.915676 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.915839 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.915960 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:26.943724 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.935467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.30.88.193:35827)
0811 05:08:26.935623 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.935628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.935774 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:08:26.935954 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.935957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.936006 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.936094 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.936746 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.937234 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.938035 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.938257 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.938360 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.940739 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.940742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.940744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.940766 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.941967 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.942405 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.942409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.942411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.942510 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.942830 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.942832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.942833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.943081 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.943201 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.943505 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.943607 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:08:26 dist-test-slave-7c35 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:26.969043 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:26.960929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.30.88.193:35827)
0811 05:08:26.961042 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:26.961045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:26.961198 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:08:26.961280 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:26.961282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:26.961320 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:26.961390 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:26.962037 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.962516 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.963360 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:26.963573 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:26.963648 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:26.965970 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:26.965974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:26.965977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:26.966005 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:26.967359 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.967841 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.967845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.967846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.967925 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:26.968234 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:26.968238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:26.968239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:26.968393 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:26.968523 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:26.968775 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 05:08:26.968887 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 05:08:26.970384 31888 tablet_service.cc:2908] Rejecting scan request for tablet c84342ef032540038740312f42b6ad28: Uninitialized: clean time has not yet been initialized
I20260811 05:08:26.972267 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31834
I20260811 05:08:26.985533 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31994
I20260811 05:08:26.993584 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 32155
I20260811 05:08:27.003116 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 31730
2026-08-11T05:08:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2581 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:27 dist-test-slave-7c35 krb5kdc[32362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:27 dist-test-slave-7c35 krb5kdc[32362](info): set up 2 sockets
Aug 11 05:08:27 dist-test-slave-7c35 krb5kdc[32362](info): commencing operation
krb5kdc: starting...
W20260811 05:08:28.133621 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:28.257025 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38753
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:28.363365 32378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:28.363621 32378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:28.363687 32378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:28.367291 32378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:28.367360 32378 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:08:28.367388 32378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:28.367430 32378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:28.367468 32378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:28.367501 32378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:28.372730 32378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38753
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:28.373715 32378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:28.374687 32378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:28.383064 32385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:28.383177 32387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:28.383064 32389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:28.383555 32378 server_base.cc:1034] running on GCE node
I20260811 05:08:28.384079 32378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:28.385094 32378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:28.386276 32378 hybrid_clock.cc:648] HybridClock initialized: now 1786424908386252 us; error 42 us; skew 500 ppm
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:28.389031 32378 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:28.389866 32378 webserver.cc:460] Webserver started at http://127.30.88.254:34809/ using document root <none> and password file <none>
I20260811 05:08:28.390439 32378 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:28.390532 32378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:28.390771 32378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:28.392845 32378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "ed06e4a4886c40daa6a4405537dd1b44"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "c7d8f7967f1a44911dd008084e9417a6"
server_key_iv: "552a42e0fff7e0133d588df0eb00c4a0"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.393424 32378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "ed06e4a4886c40daa6a4405537dd1b44"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "c7d8f7967f1a44911dd008084e9417a6"
server_key_iv: "552a42e0fff7e0133d588df0eb00c4a0"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.397043 32378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:28.399304 32398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:28.400439 32378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:28.400599 32378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "ed06e4a4886c40daa6a4405537dd1b44"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "c7d8f7967f1a44911dd008084e9417a6"
server_key_iv: "552a42e0fff7e0133d588df0eb00c4a0"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.400714 32378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:28.414803 32378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:28.417443 32378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:28.417652 32378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:28.425707 32378 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38753
I20260811 05:08:28.425714 32466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38753 every 8 connection(s)
I20260811 05:08:28.426630 32378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:28.429347 32468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:28.433077 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 32378
I20260811 05:08:28.433200 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:28.433482 31075 external_mini_cluster.cc:1428] Setting key edf2ddbc55306ebb37fa222264be3d8c
I20260811 05:08:28.435230 32468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Bootstrap starting.
I20260811 05:08:28.437685 32468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:28.438411 32468 log.cc:824] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:28.440541 32468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: No bootstrap required, opened a new log
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:28.445050 32468 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } }
I20260811 05:08:28.445353 32468 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:28.445427 32468 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed06e4a4886c40daa6a4405537dd1b44, State: Initialized, Role: FOLLOWER
I20260811 05:08:28.445940 32468 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } }
I20260811 05:08:28.446116 32468 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:28.446184 32468 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:28.446282 32468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:28.447208 32468 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } }
I20260811 05:08:28.447575 32468 leader_election.cc:302] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed06e4a4886c40daa6a4405537dd1b44; no voters:
I20260811 05:08:28.447907 32468 leader_election.cc:288] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:28.447978 32473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:28.448218 32473 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [term 1 LEADER]: Becoming Leader. State: Replica: ed06e4a4886c40daa6a4405537dd1b44, State: Running, Role: LEADER
I20260811 05:08:28.448432 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.435001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.30.88.254:38753)
0811 05:08:28.435394 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:28.435410 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:28.435444 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:08:28.436043 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:28.436060 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:28.436355 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:28.436622 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:28.437543 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.438611 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.439679 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.440117 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.440241 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.443139 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:28.443145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:28.443161 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:28.443191 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:28.445124 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.445664 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.445671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.445676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.445814 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.446206 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.446210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.446212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.446686 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:28.446925 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:28.447243 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 05:08:28.447540 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260811 05:08:28.448553 32473 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } }
I20260811 05:08:28.449184 32468 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:28.449733 32473 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed06e4a4886c40daa6a4405537dd1b44. Latest consensus state: current_term: 1 leader_uuid: "ed06e4a4886c40daa6a4405537dd1b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } } }
I20260811 05:08:28.449928 32473 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:28.450145 32474 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed06e4a4886c40daa6a4405537dd1b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e4a4886c40daa6a4405537dd1b44" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38753 } } }
I20260811 05:08:28.450313 32474 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:28.450641 32483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:28.453712 32483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:28.459419 32483 catalog_manager.cc:1347] Generated new cluster ID: 295d7da892004dd7a2f3c3163d67c98e
I20260811 05:08:28.459482 32483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:28.477658 32483 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:28.478513 32483 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:28.488754 32483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Generated new TSK 0
I20260811 05:08:28.489432 32483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:28.523514 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38753
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:28.633553 32499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:28.633795 32499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:28.633852 32499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:28.637809 32499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:28.637863 32499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:28.637899 32499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:28.637929 32499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:28.642889 32499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:38753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:28.643858 32499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:28.644829 32499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:28.654282 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:28.654314 32506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:28.654300 32499 server_base.cc:1034] running on GCE node
W20260811 05:08:28.654335 32510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:28.654901 32499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:28.655439 32499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:28.656610 32499 hybrid_clock.cc:648] HybridClock initialized: now 1786424908656591 us; error 30 us; skew 500 ppm
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:28.659790 32499 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:28.660508 32499 webserver.cc:460] Webserver started at http://127.30.88.193:41789/ using document root <none> and password file <none>
I20260811 05:08:28.660989 32499 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:28.661046 32499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:28.661197 32499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:28.663166 32499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "544627e56c3f4497a38c34cc66cf9074"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "d44a09317b034e62a047d737bba43cf7"
server_key_iv: "2e257c675f600810c54ebe718870ef19"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.663681 32499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "544627e56c3f4497a38c34cc66cf9074"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "d44a09317b034e62a047d737bba43cf7"
server_key_iv: "2e257c675f600810c54ebe718870ef19"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.667025 32499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:28.669338 32517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:28.670535 32499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:28.670653 32499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "544627e56c3f4497a38c34cc66cf9074"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "d44a09317b034e62a047d737bba43cf7"
server_key_iv: "2e257c675f600810c54ebe718870ef19"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.670754 32499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:28.684332 32499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:28.686825 32499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:28.687132 32499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:28.688506 32499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:28.688549 32499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:28.688582 32499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:28.688652 32499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:28.699609 32499 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:35791
I20260811 05:08:28.699684 32659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:35791 every 8 connection(s)
I20260811 05:08:28.700742 32499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:28.702404 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.689389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:34199 (local address 127.30.88.254:38753)
0811 05:08:28.689524 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:28.689528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:28.690525 (+ 997us) server_negotiation.cc:407] Connection header received
0811 05:08:28.691676 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:28.691681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:28.691747 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:28.691813 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:28.693414 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.694025 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.695232 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.695487 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.695601 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.698687 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:28.698691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:28.698694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:28.698721 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:28.700459 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.700930 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.700933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.700935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.701026 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.701411 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.701414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.701416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.701640 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:28.701719 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:28.702185 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 05:08:28.702286 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:08:28.703200 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.689871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.193:34199)
0811 05:08:28.690366 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:28.690405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:28.691405 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:28.692249 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:28.692259 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:28.692633 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:28.693262 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.693277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.694156 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.694159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.695102 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.695112 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.695650 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.695654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.695801 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.696353 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:28.696375 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:28.698482 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:28.700566 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.700571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.700581 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.700815 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.701131 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.701134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.701136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.701292 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.701744 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:28.701750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:28.701984 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 05:08:28.702133 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:08:28.702365 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":165,"threads_started":1}
I20260811 05:08:28.704402 32661 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38753
I20260811 05:08:28.704546 32661 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:28.705020 32661 heartbeater.cc:511] Master 127.30.88.254:38753 requested a full tablet report, sending...
I20260811 05:08:28.706573 32414 ts_manager.cc:194] Registered new tserver with Master: 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791)
I20260811 05:08:28.707897 32414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:34199
I20260811 05:08:28.709568 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 32499
I20260811 05:08:28.709692 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:28.709975 31075 external_mini_cluster.cc:1428] Setting key fe60231b512964488a6dfd1d918e16dd
I20260811 05:08:28.713639 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.706347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.193:39585)
0811 05:08:28.706631 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:28.706646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:28.706753 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:28.707009 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:28.707012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:28.707197 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:28.707306 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.707313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.708081 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.708085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.708852 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.708863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.709308 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.709311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.709411 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.709903 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:28.709916 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:28.710290 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:28.712188 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.712190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.712192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.712383 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.712675 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.712677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.712678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.712749 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.713187 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:28.713188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:28.713254 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 05:08:28.713328 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:08:28.713455 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260811 05:08:28.713781 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.706456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:39585 (local address 127.30.88.254:38753)
0811 05:08:28.706560 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:28.706566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:28.706772 (+ 206us) server_negotiation.cc:407] Connection header received
0811 05:08:28.706823 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:28.706825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:28.706897 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:28.706973 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:28.707435 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.707901 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.708985 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.709196 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.709320 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.710436 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:28.710439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:28.710441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:28.710463 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:28.712078 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.712480 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.712487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.712491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.712582 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.712836 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.712842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.712847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.713084 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:28.713196 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:28.713411 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 05:08:28.713552 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:28.738658 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38753
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:28.847076 32671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:28.847313 32671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:28.847378 32671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:28.851500 32671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:28.851552 32671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:28.851617 32671 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:28.851703 32671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:28.856637 32671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:38753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:28.857676 32671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:28.858656 32671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:28.868146 32678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:28.868186 32677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:28.868165 32671 server_base.cc:1034] running on GCE node
W20260811 05:08:28.868186 32682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:28.868773 32671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:28.869387 32671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:28.870575 32671 hybrid_clock.cc:648] HybridClock initialized: now 1786424908870556 us; error 40 us; skew 500 ppm
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:28.873423 32671 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:28.874357 32671 webserver.cc:460] Webserver started at http://127.30.88.194:39701/ using document root <none> and password file <none>
I20260811 05:08:28.875056 32671 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:28.875149 32671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:28.875383 32671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:28.877436 32671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "40c3895b16644162bf871d272a7b8ade"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "5577011fa2e3d180c9231e11fbfc0abb"
server_key_iv: "5bd0b82a5e63560786d7f6764fb9ae67"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.878021 32671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "40c3895b16644162bf871d272a7b8ade"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "5577011fa2e3d180c9231e11fbfc0abb"
server_key_iv: "5bd0b82a5e63560786d7f6764fb9ae67"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.881515 32671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 05:08:28.883814 32690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:28.884970 32671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:28.885104 32671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "40c3895b16644162bf871d272a7b8ade"
format_stamp: "Formatted at 2026-08-11 05:08:28 on dist-test-slave-7c35"
server_key: "5577011fa2e3d180c9231e11fbfc0abb"
server_key_iv: "5bd0b82a5e63560786d7f6764fb9ae67"
server_key_version: "encryptionkey@0"
I20260811 05:08:28.885210 32671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:28.921943 32671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:28.925146 32671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:28.925349 32671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:28.926719 32671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:28.926790 32671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:28.926888 32671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:28.926934 32671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:28 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:28.937202 32671 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:33537
I20260811 05:08:28.937294 360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:33537 every 8 connection(s)
I20260811 05:08:28.938064 32671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:28.940946 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.927607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:53429 (local address 127.30.88.254:38753)
0811 05:08:28.927798 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:28.927805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:28.928593 (+ 788us) server_negotiation.cc:407] Connection header received
0811 05:08:28.929744 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:28.929749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:28.929814 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:28.929915 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:28.931459 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.932361 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.934131 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.934498 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.934633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.937481 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:28.937484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:28.937487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:28.937519 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:28.939161 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.939697 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.939700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.939701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.939777 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.940036 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.940038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.940039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.940202 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:28.940276 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:28.940724 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 05:08:28.940820 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 05:08:28.941715 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.927972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.194:53429)
0811 05:08:28.928429 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:28.928477 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:28.929503 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:28.930109 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:28.930136 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:28.930576 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:28.931272 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.931294 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.932532 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.932537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.933990 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.934003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.934650 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.934653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.934797 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.935446 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:28.935472 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:28.937295 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:28.939283 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.939288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.939301 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.939605 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.939840 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.939842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.939844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.939957 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.940290 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:28.940295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:28.940522 (+ 227us) client_negotiation.cc:769] Sending connection context
0811 05:08:28.940667 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:08:28.940893 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260811 05:08:28.942888 361 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38753
I20260811 05:08:28.943051 361 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:28.943579 361 heartbeater.cc:511] Master 127.30.88.254:38753 requested a full tablet report, sending...
I20260811 05:08:28.944576 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 32671
I20260811 05:08:28.944587 32414 ts_manager.cc:194] Registered new tserver with Master: 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537)
I20260811 05:08:28.944692 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:28.944984 31075 external_mini_cluster.cc:1428] Setting key 7f5d2b3588c9fbaae309343bd1d62091
I20260811 05:08:28.945241 32414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:53429
I20260811 05:08:28.953747 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.945172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.194:39443)
0811 05:08:28.945671 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:28.945686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:28.945789 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:28.946020 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:28.946023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:28.946225 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:28.946343 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.946350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.947394 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.947401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.948382 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:28.948397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.949059 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.949065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:28.949185 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.949683 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:28.949702 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:28.950214 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:28.952258 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.952261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.952262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.952461 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.952770 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:28.952772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:28.952773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:28.952847 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:28.953334 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:28.953336 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:28.953409 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 05:08:28.953499 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 05:08:28.953603 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":90,"threads_started":1}
I20260811 05:08:28.953956 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:28.945270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:39443 (local address 127.30.88.254:38753)
0811 05:08:28.945570 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:28.945574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:28.945763 (+ 189us) server_negotiation.cc:407] Connection header received
0811 05:08:28.945865 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:28.945868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:28.945908 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:28.946004 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:28.946455 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.947231 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.948532 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:28.948928 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:28.949046 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:28.950371 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:28.950374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:28.950377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:28.950429 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:28.952136 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.952566 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.952572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.952577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.952674 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:28.952944 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:28.952950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:28.952955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:28.953236 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:28.953346 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:28.953597 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 05:08:28.953735 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:28.973999 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38753
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:29.083297 372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:29.083556 372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:29.083626 372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:29.087574 372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:29.087643 372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:29.087704 372 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:29.087782 372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:29.092933 372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:38753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:29.093777 372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:29.094652 372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:29.103847 377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:29.103855 382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:29.104043 380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:29.104332 372 server_base.cc:1034] running on GCE node
I20260811 05:08:29.104733 372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:29.105355 372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:29.106523 372 hybrid_clock.cc:648] HybridClock initialized: now 1786424909106509 us; error 30 us; skew 500 ppm
Aug 11 05:08:29 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424909, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:29.109190 372 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:29.109974 372 webserver.cc:460] Webserver started at http://127.30.88.195:33461/ using document root <none> and password file <none>
I20260811 05:08:29.110487 372 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:29.110544 372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:29.110697 372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:29.112613 372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "564adeb87da8408baee87d42a6ffdbd1"
format_stamp: "Formatted at 2026-08-11 05:08:29 on dist-test-slave-7c35"
server_key: "c03b287904de3e298325530b21ec78d7"
server_key_iv: "e7cef419b8cf9452fe4a1a3984931f71"
server_key_version: "encryptionkey@0"
I20260811 05:08:29.113129 372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "564adeb87da8408baee87d42a6ffdbd1"
format_stamp: "Formatted at 2026-08-11 05:08:29 on dist-test-slave-7c35"
server_key: "c03b287904de3e298325530b21ec78d7"
server_key_iv: "e7cef419b8cf9452fe4a1a3984931f71"
server_key_version: "encryptionkey@0"
I20260811 05:08:29.116564 372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:29.118841 390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:29.120087 372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:29.120224 372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "564adeb87da8408baee87d42a6ffdbd1"
format_stamp: "Formatted at 2026-08-11 05:08:29 on dist-test-slave-7c35"
server_key: "c03b287904de3e298325530b21ec78d7"
server_key_iv: "e7cef419b8cf9452fe4a1a3984931f71"
server_key_version: "encryptionkey@0"
I20260811 05:08:29.120330 372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:29.136924 372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:29.140107 372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:29.140290 372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:29.141582 372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:29.141652 372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:29.141738 372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:29.141781 372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:29 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424909, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:29.152688 372 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:35163
I20260811 05:08:29.152694 526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:35163 every 8 connection(s)
I20260811 05:08:29.153855 372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:29.156008 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.142437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:34289 (local address 127.30.88.254:38753)
0811 05:08:29.142582 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.142586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.143632 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 05:08:29.144748 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.144751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.144844 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.144909 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:29.146636 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.147576 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.149003 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.149240 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.149359 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.152395 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:29.152400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:29.152406 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:29.152440 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:29.154187 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.154759 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.154763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.154764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.154875 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.155169 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.155171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.155172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.155336 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:29.155403 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.155806 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.155895 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:08:29.156699 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.142968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.195:34289)
0811 05:08:29.143473 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.143513 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.144475 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.145346 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.145355 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.145774 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:29.146445 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.146465 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.147758 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.147763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.148859 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.148869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.149508 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.149511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.149665 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.150229 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:29.150244 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:29.152183 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:29.154318 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.154324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.154336 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.154663 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.154964 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.154967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.154968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.155096 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.155417 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:29.155421 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:29.155622 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.155756 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.155964 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":152,"threads_started":1}
I20260811 05:08:29.157806 527 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38753
I20260811 05:08:29.158010 527 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:29.158478 527 heartbeater.cc:511] Master 127.30.88.254:38753 requested a full tablet report, sending...
I20260811 05:08:29.159526 32414 ts_manager.cc:194] Registered new tserver with Master: 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163)
I20260811 05:08:29.160166 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 372
I20260811 05:08:29.160234 32414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:34289
I20260811 05:08:29.160298 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:29.160606 31075 external_mini_cluster.cc:1428] Setting key ea1102532ef41403a90f79210bc652fd
I20260811 05:08:29.162627 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:08:29.168442 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.159973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:52001 (local address 127.30.88.254:38753)
0811 05:08:29.160082 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.160086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.161166 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 05:08:29.161261 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.161264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.161308 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.161384 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:29.161822 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.162506 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.163461 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.163800 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.163910 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.164932 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:29.164940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:29.164947 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:29.164984 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:29.166424 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.166953 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.166958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.166959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.167064 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.167484 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.167488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.167490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.167703 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:29.167840 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.168142 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.168263 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:08:29.168489 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.159857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.30.88.195:52001)
0811 05:08:29.161065 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.161079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.161186 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.161410 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.161413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.161607 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:29.161713 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.161719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.162667 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.162671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.163347 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.163356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.163915 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.163917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.163986 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.164470 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:29.164481 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:29.164757 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:29.166559 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.166565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.166569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.166827 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.167253 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.167256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.167257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.167358 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.167822 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:29.167826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:29.167941 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.168077 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.168238 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"thread_start_us":114,"threads_started":1}
Aug 11 05:08:29 dist-test-slave-7c35 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:08:29 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:29.191713 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.183278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.30.88.254:38753)
0811 05:08:29.183420 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.183424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.183435 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:08:29.183481 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.183483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.183518 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.183611 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:29.184297 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.184801 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.185664 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.185889 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.185999 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.188461 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:29.188464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:29.188466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:29.188489 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:29.189949 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.190392 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.190396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.190397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.190488 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.190774 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.190778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.190780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.191021 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:29.191166 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.191423 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.191547 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:08:29.194926 32414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:29.197745 32414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:29.211598 546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.205805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35791 (local address 127.0.0.1:60656)
0811 05:08:29.206365 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.206413 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.206540 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.207714 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.207717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.207734 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.207861 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.207868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.208974 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.208977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.210182 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.210193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.211073 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.211077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.211188 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.211227 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.211343 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.211454 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":77,"threads_started":1}
I20260811 05:08:29.211639 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.205939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.30.88.193:35791)
0811 05:08:29.207248 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.207255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.207277 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:08:29.207355 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.207359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.207559 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.208009 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.208137 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.208845 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.210313 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.210941 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.211056 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.211198 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.211389 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.211495 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":78,"threads_started":1}
I20260811 05:08:29.212236 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.205938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33537 (local address 127.0.0.1:55438)
0811 05:08:29.207124 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.207138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.207241 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.207690 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.207696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.207717 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.207850 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.207856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.209336 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.209339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.210730 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.210742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.211850 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.211854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.211941 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.211963 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.212032 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.212103 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":81,"spinlock_wait_cycles":3328,"thread_start_us":91,"threads_started":1}
I20260811 05:08:29.212283 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.206578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.30.88.194:33537)
0811 05:08:29.207078 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.207090 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.207235 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:08:29.207355 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.207361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.207539 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.207699 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.207968 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.209205 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.210905 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.211741 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.211851 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.212020 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.212070 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.212153 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":166,"threads_started":1}
I20260811 05:08:29.213085 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.206186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:35163 (local address 127.0.0.1:37928)
0811 05:08:29.207244 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.207258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.207353 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.207846 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.207849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.207862 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.207949 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.207969 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.209336 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.209339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.210730 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.210742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.212039 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.212042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.212138 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.212162 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.212230 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.212302 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"mutex_wait_us":146,"thread_start_us":91,"threads_started":1}
I20260811 05:08:29.213454 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.206603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.30.88.195:35163)
0811 05:08:29.206980 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.206992 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.207361 (+ 369us) server_negotiation.cc:407] Connection header received
0811 05:08:29.207437 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.207441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.207579 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.207714 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.208079 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.209193 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.210901 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.211709 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.211816 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.211969 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.212476 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.213334 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":151,"threads_started":1}
I20260811 05:08:29.215008 32750 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f (DEFAULT_TABLE table=test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.214998 32581 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f (DEFAULT_TABLE table=test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.215730 430 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f (DEFAULT_TABLE table=test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.216094 32581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0151ee490447c2a3759564fa0d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.216092 32750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0151ee490447c2a3759564fa0d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.216655 430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0151ee490447c2a3759564fa0d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.222894 552 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Bootstrap starting.
I20260811 05:08:29.224222 554 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Bootstrap starting.
I20260811 05:08:29.224223 553 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Bootstrap starting.
I20260811 05:08:29.225148 552 tablet_bootstrap.cc:654] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.225896 552 log.cc:824] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:29.226342 554 tablet_bootstrap.cc:654] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.226342 553 tablet_bootstrap.cc:654] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.227218 554 log.cc:824] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:29.227218 553 log.cc:824] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:29.228401 552 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: No bootstrap required, opened a new log
I20260811 05:08:29.228617 552 ts_tablet_manager.cc:1397] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 05:08:29.228933 553 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: No bootstrap required, opened a new log
I20260811 05:08:29.229120 553 ts_tablet_manager.cc:1397] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:08:29.231808 552 raft_consensus.cc:348] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.232095 552 raft_consensus.cc:374] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.232157 554 tablet_bootstrap.cc:492] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: No bootstrap required, opened a new log
I20260811 05:08:29.232156 552 raft_consensus.cc:729] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564adeb87da8408baee87d42a6ffdbd1, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.232317 554 ts_tablet_manager.cc:1397] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 05:08:29.232261 553 raft_consensus.cc:348] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.232506 553 raft_consensus.cc:374] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.232563 553 raft_consensus.cc:729] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544627e56c3f4497a38c34cc66cf9074, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.232620 552 consensus_queue.cc:260] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.232983 553 consensus_queue.cc:260] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.233361 527 heartbeater.cc:503] Master 127.30.88.254:38753 was elected leader, sending a full tablet report...
I20260811 05:08:29.233866 553 ts_tablet_manager.cc:1428] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:08:29.234170 32661 heartbeater.cc:503] Master 127.30.88.254:38753 was elected leader, sending a full tablet report...
I20260811 05:08:29.235100 552 ts_tablet_manager.cc:1428] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 05:08:29.235821 554 raft_consensus.cc:348] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.236001 554 raft_consensus.cc:374] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.236083 554 raft_consensus.cc:729] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c3895b16644162bf871d272a7b8ade, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.236524 554 consensus_queue.cc:260] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.237180 361 heartbeater.cc:503] Master 127.30.88.254:38753 was elected leader, sending a full tablet report...
I20260811 05:08:29.237489 554 ts_tablet_manager.cc:1428] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 05:08:29.404965 528 tablet.cc:2369] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:29.429349 562 raft_consensus.cc:482] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:29.429538 562 raft_consensus.cc:504] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.430701 562 leader_election.cc:288] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791), 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537)
I20260811 05:08:29.435227 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.431225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:49297 (local address 127.30.88.194:33537)
0811 05:08:29.431347 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.431351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.431553 (+ 202us) server_negotiation.cc:407] Connection header received
0811 05:08:29.431694 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.431697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.431757 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.431828 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.432612 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.433156 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.434204 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.434615 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.434763 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.434824 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.435029 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.435107 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:08:29.435253 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.431062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33537 (local address 127.30.88.195:49297)
0811 05:08:29.431472 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.431484 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.431582 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.431905 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.431908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.431928 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.432477 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.432485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.433282 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.433286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.434078 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.434087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.434762 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.434765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.434902 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.434921 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.435013 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.435095 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":2,"thread_start_us":130,"threads_started":1}
I20260811 05:08:29.435546 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.430965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35791 (local address 127.30.88.195:44819)
0811 05:08:29.431379 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.431396 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.431515 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.431804 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.431810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.431831 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.431972 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.431978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.432776 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.432779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.434096 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.434107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.435143 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.435146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.435262 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.435283 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.435350 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.435412 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":40}
I20260811 05:08:29.435920 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0151ee490447c2a3759564fa0d8a3f" candidate_uuid: "564adeb87da8408baee87d42a6ffdbd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c3895b16644162bf871d272a7b8ade" is_pre_election: true
I20260811 05:08:29.436013 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.431036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:44819 (local address 127.30.88.193:35791)
0811 05:08:29.431207 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.431212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.431472 (+ 260us) server_negotiation.cc:407] Connection header received
0811 05:08:29.431635 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.431638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.431690 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.431772 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.432114 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.432662 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.434241 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.434907 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.435033 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.435080 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.435797 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.435881 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 05:08:29.436250 304 raft_consensus.cc:2454] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 564adeb87da8408baee87d42a6ffdbd1 in term 0.
I20260811 05:08:29.436512 32604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0151ee490447c2a3759564fa0d8a3f" candidate_uuid: "564adeb87da8408baee87d42a6ffdbd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544627e56c3f4497a38c34cc66cf9074" is_pre_election: true
I20260811 05:08:29.436774 32604 raft_consensus.cc:2454] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 564adeb87da8408baee87d42a6ffdbd1 in term 0.
I20260811 05:08:29.436789 393 leader_election.cc:302] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c3895b16644162bf871d272a7b8ade, 564adeb87da8408baee87d42a6ffdbd1; no voters:
I20260811 05:08:29.437090 562 raft_consensus.cc:2781] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:29.437192 562 raft_consensus.cc:482] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:29.437248 562 raft_consensus.cc:3037] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.438283 562 raft_consensus.cc:504] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.438658 562 leader_election.cc:288] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [CANDIDATE]: Term 1 election: Requested vote from peers 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791), 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537)
I20260811 05:08:29.439038 32604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0151ee490447c2a3759564fa0d8a3f" candidate_uuid: "564adeb87da8408baee87d42a6ffdbd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544627e56c3f4497a38c34cc66cf9074"
I20260811 05:08:29.439090 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0151ee490447c2a3759564fa0d8a3f" candidate_uuid: "564adeb87da8408baee87d42a6ffdbd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c3895b16644162bf871d272a7b8ade"
I20260811 05:08:29.439162 32604 raft_consensus.cc:3037] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.439193 304 raft_consensus.cc:3037] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Advancing to term 1
W20260811 05:08:29.439242 362 tablet.cc:2369] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:29.440433 304 raft_consensus.cc:2454] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 564adeb87da8408baee87d42a6ffdbd1 in term 1.
I20260811 05:08:29.440433 32604 raft_consensus.cc:2454] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 564adeb87da8408baee87d42a6ffdbd1 in term 1.
I20260811 05:08:29.440780 391 leader_election.cc:302] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 544627e56c3f4497a38c34cc66cf9074, 564adeb87da8408baee87d42a6ffdbd1; no voters:
I20260811 05:08:29.440958 562 raft_consensus.cc:2781] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:29.441190 562 raft_consensus.cc:686] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 1 LEADER]: Becoming Leader. State: Replica: 564adeb87da8408baee87d42a6ffdbd1, State: Running, Role: LEADER
I20260811 05:08:29.441516 562 consensus_queue.cc:237] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.444612 32415 catalog_manager.cc:5495] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "564adeb87da8408baee87d42a6ffdbd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } health_report { overall_health: HEALTHY } } }
W20260811 05:08:29.452180 32662 tablet.cc:2369] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:29.505260 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.501785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.30.88.195:35163)
0811 05:08:29.501919 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.501923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.501937 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:29.502135 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.502138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.502191 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.502273 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:29.502513 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.503063 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.503847 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.504082 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.504165 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.504430 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.504828 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.504919 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.505062 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.505108 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:08:29.510110 32604 raft_consensus.cc:1261] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Refusing update from remote peer 564adeb87da8408baee87d42a6ffdbd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.510113 304 raft_consensus.cc:1261] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Refusing update from remote peer 564adeb87da8408baee87d42a6ffdbd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.510887 567 consensus_queue.cc:1035] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.511080 562 consensus_queue.cc:1035] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.517443 571 mvcc.cc:205] Tried to move back new op lower bound from 7317196429347471360 to 7317196429075066880. Current Snapshot: MvccSnapshot[applied={T|T < 7317196429347471360}]
I20260811 05:08:29.528874 32414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:29.528999 32414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:29.531339 32414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Sending DeleteTablet for 3 replicas of tablet 7f0151ee490447c2a3759564fa0d8a3f
I20260811 05:08:29.532119 32581 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60656
I20260811 05:08:29.532328 430 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:37928
I20260811 05:08:29.532549 32750 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0151ee490447c2a3759564fa0d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55438
I20260811 05:08:29.532752 584 tablet_replica.cc:330] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: stopping tablet replica
I20260811 05:08:29.533056 585 tablet_replica.cc:330] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: stopping tablet replica
I20260811 05:08:29.533043 584 raft_consensus.cc:2229] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:29.533151 32414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:08:29.533350 584 raft_consensus.cc:2258] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:29.533593 585 raft_consensus.cc:2229] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 1 LEADER]: Raft consensus shutting down.
W20260811 05:08:29.533669 32414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:29.534045 585 raft_consensus.cc:2258] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:29.535014 587 tablet_replica.cc:330] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: stopping tablet replica
I20260811 05:08:29.535250 587 raft_consensus.cc:2229] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:29.535486 587 raft_consensus.cc:2258] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:29.535657 585 ts_tablet_manager.cc:1905] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:29.536655 587 ts_tablet_manager.cc:1905] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:29.538753 585 ts_tablet_manager.cc:1918] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:29.538930 585 log.cc:1196] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/7f0151ee490447c2a3759564fa0d8a3f
I20260811 05:08:29.539189 587 ts_tablet_manager.cc:1918] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:29.539281 587 log.cc:1196] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/7f0151ee490447c2a3759564fa0d8a3f
I20260811 05:08:29.539356 430 tablet_service.cc:1467] Processing CreateTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 (DEFAULT_TABLE table=test-table [id=d1493ddce59f43beaa66b79565544ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.539511 585 ts_tablet_manager.cc:1939] T 7f0151ee490447c2a3759564fa0d8a3f P 564adeb87da8408baee87d42a6ffdbd1: Deleting consensus metadata
I20260811 05:08:29.539589 587 ts_tablet_manager.cc:1939] T 7f0151ee490447c2a3759564fa0d8a3f P 40c3895b16644162bf871d272a7b8ade: Deleting consensus metadata
I20260811 05:08:29.539654 430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bd025c718d24c8d8e95427c5924ac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.541527 552 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Bootstrap starting.
I20260811 05:08:29.541956 32402 catalog_manager.cc:4841] TS 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163): tablet 7f0151ee490447c2a3759564fa0d8a3f (table test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]) successfully deleted
I20260811 05:08:29.542011 32581 tablet_service.cc:1467] Processing CreateTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 (DEFAULT_TABLE table=test-table [id=d1493ddce59f43beaa66b79565544ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.542253 32581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bd025c718d24c8d8e95427c5924ac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.542347 32401 catalog_manager.cc:4841] TS 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537): tablet 7f0151ee490447c2a3759564fa0d8a3f (table test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]) successfully deleted
I20260811 05:08:29.542567 552 tablet_bootstrap.cc:654] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.542791 32750 tablet_service.cc:1467] Processing CreateTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 (DEFAULT_TABLE table=test-table [id=d1493ddce59f43beaa66b79565544ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:29.543062 32750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bd025c718d24c8d8e95427c5924ac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.543780 552 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: No bootstrap required, opened a new log
I20260811 05:08:29.543850 552 ts_tablet_manager.cc:1397] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:29.544302 552 raft_consensus.cc:348] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.544431 552 raft_consensus.cc:374] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.544471 552 raft_consensus.cc:729] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564adeb87da8408baee87d42a6ffdbd1, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.544600 552 consensus_queue.cc:260] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.544864 552 ts_tablet_manager.cc:1428] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:29.545428 554 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Bootstrap starting.
I20260811 05:08:29.545495 553 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Bootstrap starting.
I20260811 05:08:29.546270 554 tablet_bootstrap.cc:654] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.546482 553 tablet_bootstrap.cc:654] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.547808 584 ts_tablet_manager.cc:1905] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:29.547925 554 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: No bootstrap required, opened a new log
I20260811 05:08:29.548019 554 ts_tablet_manager.cc:1397] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:29.548542 554 raft_consensus.cc:348] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.548655 554 raft_consensus.cc:374] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.548684 554 raft_consensus.cc:729] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c3895b16644162bf871d272a7b8ade, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.548800 554 consensus_queue.cc:260] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.549039 554 ts_tablet_manager.cc:1428] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:29.550801 584 ts_tablet_manager.cc:1918] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:29.550958 584 log.cc:1196] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/7f0151ee490447c2a3759564fa0d8a3f
I20260811 05:08:29.551337 584 ts_tablet_manager.cc:1939] T 7f0151ee490447c2a3759564fa0d8a3f P 544627e56c3f4497a38c34cc66cf9074: Deleting consensus metadata
I20260811 05:08:29.551677 553 tablet_bootstrap.cc:492] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: No bootstrap required, opened a new log
I20260811 05:08:29.551759 553 ts_tablet_manager.cc:1397] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 05:08:29.552372 553 raft_consensus.cc:348] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.552484 553 raft_consensus.cc:374] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.552521 553 raft_consensus.cc:729] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544627e56c3f4497a38c34cc66cf9074, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.552597 32399 catalog_manager.cc:4841] TS 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791): tablet 7f0151ee490447c2a3759564fa0d8a3f (table test-table [id=bd7cc20b9368431bad7fd1fb74d26af6]) successfully deleted
I20260811 05:08:29.552651 553 consensus_queue.cc:260] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.552944 553 ts_tablet_manager.cc:1428] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:29.590653 565 raft_consensus.cc:482] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:29.590760 565 raft_consensus.cc:504] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.591939 565 leader_election.cc:288] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791), 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163)
I20260811 05:08:29.597064 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.592220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:50505 (local address 127.30.88.193:35791)
0811 05:08:29.592346 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.592353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.592526 (+ 173us) server_negotiation.cc:407] Connection header received
0811 05:08:29.592733 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.592736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.592785 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.592836 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.593246 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.593998 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.595927 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.596541 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.596665 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.596726 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.596883 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.596957 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:08:29.597096 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.592133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35791 (local address 127.30.88.194:50505)
0811 05:08:29.592440 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.592453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.592604 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.592995 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.592997 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.593009 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.593116 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.593121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.594149 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.594153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.595794 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.595806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.596655 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.596657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.596749 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.596770 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.596861 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.596933 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":188,"spinlock_wait_cycles":4992,"thread_start_us":95,"threads_started":1}
I20260811 05:08:29.597693 32604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bd025c718d24c8d8e95427c5924ac88" candidate_uuid: "40c3895b16644162bf871d272a7b8ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544627e56c3f4497a38c34cc66cf9074" is_pre_election: true
I20260811 05:08:29.597847 32604 raft_consensus.cc:2454] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c3895b16644162bf871d272a7b8ade in term 0.
I20260811 05:08:29.597977 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.592197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:35163 (local address 127.30.88.194:44083)
0811 05:08:29.592456 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.592469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.592604 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.592921 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.592924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.592946 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.593095 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.593103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.594149 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.594153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.595794 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.595806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.597490 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.597493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.597591 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.597614 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.597759 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.597841 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":165,"spinlock_wait_cycles":17024,"thread_start_us":99,"threads_started":1}
I20260811 05:08:29.597973 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.592562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:44083 (local address 127.30.88.195:35163)
0811 05:08:29.592676 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.592680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.592693 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:29.592747 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.592750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.592794 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.592886 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.593247 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.593998 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.596786 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.597367 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.597490 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.597538 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.597802 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.597873 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:08:29.598246 32691 leader_election.cc:302] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c3895b16644162bf871d272a7b8ade, 544627e56c3f4497a38c34cc66cf9074; no voters:
I20260811 05:08:29.598531 565 raft_consensus.cc:2781] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:29.598595 565 raft_consensus.cc:482] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:29.598639 565 raft_consensus.cc:3037] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.599047 472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bd025c718d24c8d8e95427c5924ac88" candidate_uuid: "40c3895b16644162bf871d272a7b8ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564adeb87da8408baee87d42a6ffdbd1" is_pre_election: true
I20260811 05:08:29.599274 472 raft_consensus.cc:2454] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c3895b16644162bf871d272a7b8ade in term 0.
I20260811 05:08:29.599932 565 raft_consensus.cc:504] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.600260 565 leader_election.cc:288] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [CANDIDATE]: Term 1 election: Requested vote from peers 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791), 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163)
I20260811 05:08:29.600545 32604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bd025c718d24c8d8e95427c5924ac88" candidate_uuid: "40c3895b16644162bf871d272a7b8ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544627e56c3f4497a38c34cc66cf9074"
I20260811 05:08:29.600607 472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bd025c718d24c8d8e95427c5924ac88" candidate_uuid: "40c3895b16644162bf871d272a7b8ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564adeb87da8408baee87d42a6ffdbd1"
I20260811 05:08:29.600725 472 raft_consensus.cc:3037] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.600723 32604 raft_consensus.cc:3037] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.601598 32604 raft_consensus.cc:2454] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c3895b16644162bf871d272a7b8ade in term 1.
I20260811 05:08:29.601871 32691 leader_election.cc:302] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c3895b16644162bf871d272a7b8ade, 544627e56c3f4497a38c34cc66cf9074; no voters:
I20260811 05:08:29.601898 472 raft_consensus.cc:2454] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c3895b16644162bf871d272a7b8ade in term 1.
I20260811 05:08:29.602100 565 raft_consensus.cc:2781] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:29.602339 565 raft_consensus.cc:686] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 1 LEADER]: Becoming Leader. State: Replica: 40c3895b16644162bf871d272a7b8ade, State: Running, Role: LEADER
I20260811 05:08:29.602587 565 consensus_queue.cc:237] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.604923 32415 catalog_manager.cc:5495] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade reported cstate change: term changed from 0 to 1, leader changed from <none> to 40c3895b16644162bf871d272a7b8ade (127.30.88.194). New cstate: current_term: 1 leader_uuid: "40c3895b16644162bf871d272a7b8ade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } health_report { overall_health: UNKNOWN } } }
Aug 11 05:08:29 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:29.649977 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.640712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.30.88.254:38753)
0811 05:08:29.640861 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.640867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.641049 (+ 182us) server_negotiation.cc:407] Connection header received
0811 05:08:29.641217 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.641220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.641263 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.641343 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:29.641970 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.642579 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.643467 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.643744 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.643879 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.646502 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:29.646506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:29.646509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:29.646538 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:29.647974 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.648553 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.648558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.648560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.648639 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:29.648918 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:29.648921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:29.648923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:29.649156 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:29.649272 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.649669 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.649815 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":44}
I20260811 05:08:29.649978 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.640651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38753 (local address 127.0.0.1:53774)
0811 05:08:29.640962 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.640973 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.641105 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.641365 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.641367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.641755 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:29.641843 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.641849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.642683 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.642689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.643344 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.643354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.643886 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.643897 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.644001 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.644661 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:29.644674 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:29.646348 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:29.648094 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.648106 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.648111 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.648452 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.648732 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:29.648737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:29.648739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:29.648826 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:29.649272 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:29.649280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:29.649468 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.649604 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.649763 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":17,"thread_start_us":125,"threads_started":1}
I20260811 05:08:29.655264 32415 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:53774:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:08:29.656096 32415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:29.661351 32581 tablet_service.cc:1467] Processing CreateTablet for tablet c3568b48adae4662b15cc0534c208522 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=154ea222796c4cb98a4fbcae318a2251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:29.661782 32581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3568b48adae4662b15cc0534c208522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.661648 430 tablet_service.cc:1467] Processing CreateTablet for tablet c3568b48adae4662b15cc0534c208522 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=154ea222796c4cb98a4fbcae318a2251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:29.662061 430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3568b48adae4662b15cc0534c208522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.664366 553 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: Bootstrap starting.
I20260811 05:08:29.664748 32750 tablet_service.cc:1467] Processing CreateTablet for tablet c3568b48adae4662b15cc0534c208522 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=154ea222796c4cb98a4fbcae318a2251]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:29.665125 32750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3568b48adae4662b15cc0534c208522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:29.665256 552 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: Bootstrap starting.
I20260811 05:08:29.665447 553 tablet_bootstrap.cc:654] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.666282 552 tablet_bootstrap.cc:654] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.667578 552 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: No bootstrap required, opened a new log
I20260811 05:08:29.667701 552 ts_tablet_manager.cc:1397] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.002s
I20260811 05:08:29.667943 553 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: No bootstrap required, opened a new log
I20260811 05:08:29.668044 553 ts_tablet_manager.cc:1397] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 05:08:29.668138 552 raft_consensus.cc:348] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.668249 552 raft_consensus.cc:374] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.668288 552 raft_consensus.cc:729] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564adeb87da8408baee87d42a6ffdbd1, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.668424 552 consensus_queue.cc:260] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.668499 554 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: Bootstrap starting.
I20260811 05:08:29.668480 553 raft_consensus.cc:348] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.668592 553 raft_consensus.cc:374] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.668633 553 raft_consensus.cc:729] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544627e56c3f4497a38c34cc66cf9074, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.668749 553 consensus_queue.cc:260] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.669032 553 ts_tablet_manager.cc:1428] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:08:29.669046 580 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.669183 580 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.669514 554 tablet_bootstrap.cc:654] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:29.670226 552 ts_tablet_manager.cc:1428] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:08:29.670244 564 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.670392 564 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.672086 554 tablet_bootstrap.cc:492] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: No bootstrap required, opened a new log
I20260811 05:08:29.672194 554 ts_tablet_manager.cc:1397] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 05:08:29.672606 554 raft_consensus.cc:348] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.672729 554 raft_consensus.cc:374] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:29.672767 554 raft_consensus.cc:729] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c3895b16644162bf871d272a7b8ade, State: Initialized, Role: FOLLOWER
I20260811 05:08:29.672895 554 consensus_queue.cc:260] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.673166 554 ts_tablet_manager.cc:1428] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:29.673401 565 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.673520 565 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.853603 564 raft_consensus.cc:482] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:29.853734 564 raft_consensus.cc:504] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.855026 564 leader_election.cc:288] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537), 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163)
I20260811 05:08:29.858991 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.855522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:60505 (local address 127.30.88.194:33537)
0811 05:08:29.855733 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.855739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.855760 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:08:29.855818 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.855821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.855877 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.855949 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.856268 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.856792 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.857946 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.858395 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.858518 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.858579 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.858738 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.858824 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 05:08:29.859022 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.855584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:36593 (local address 127.30.88.195:35163)
0811 05:08:29.855832 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.855837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.855871 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:08:29.855951 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.855954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.856003 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.856076 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:29.856346 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.856820 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.858046 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.858437 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.858558 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.858611 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.858827 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.858914 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 05:08:29.859019 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.855201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33537 (local address 127.30.88.193:60505)
0811 05:08:29.855581 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.855613 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.855714 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.855989 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.855992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.856010 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.856131 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.856141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.856907 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.856914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.857826 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.857836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.858504 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.858508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.858591 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.858621 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.858721 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.858820 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":116,"threads_started":1}
I20260811 05:08:29.859027 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.855270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:35163 (local address 127.30.88.193:36593)
0811 05:08:29.855709 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.855724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.855821 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.856116 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.856118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.856133 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:29.856226 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.856232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.856978 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.856980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.857922 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.857936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.858563 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.858566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.858684 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.858717 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.858794 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.858873 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260811 05:08:29.859439 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3568b48adae4662b15cc0534c208522" candidate_uuid: "544627e56c3f4497a38c34cc66cf9074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c3895b16644162bf871d272a7b8ade" is_pre_election: true
I20260811 05:08:29.859587 304 raft_consensus.cc:2454] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 544627e56c3f4497a38c34cc66cf9074 in term 0.
I20260811 05:08:29.859923 32522 leader_election.cc:302] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c3895b16644162bf871d272a7b8ade, 544627e56c3f4497a38c34cc66cf9074; no voters:
I20260811 05:08:29.859946 472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3568b48adae4662b15cc0534c208522" candidate_uuid: "544627e56c3f4497a38c34cc66cf9074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564adeb87da8408baee87d42a6ffdbd1" is_pre_election: true
I20260811 05:08:29.860054 472 raft_consensus.cc:2454] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 544627e56c3f4497a38c34cc66cf9074 in term 0.
I20260811 05:08:29.860198 564 raft_consensus.cc:2781] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:29.860255 564 raft_consensus.cc:482] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:29.860302 564 raft_consensus.cc:3037] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.861269 564 raft_consensus.cc:504] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.861598 564 leader_election.cc:288] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [CANDIDATE]: Term 1 election: Requested vote from peers 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537), 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163)
I20260811 05:08:29.861900 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3568b48adae4662b15cc0534c208522" candidate_uuid: "544627e56c3f4497a38c34cc66cf9074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c3895b16644162bf871d272a7b8ade"
I20260811 05:08:29.861994 304 raft_consensus.cc:3037] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.862010 472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3568b48adae4662b15cc0534c208522" candidate_uuid: "544627e56c3f4497a38c34cc66cf9074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564adeb87da8408baee87d42a6ffdbd1"
I20260811 05:08:29.862179 472 raft_consensus.cc:3037] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:29.863044 304 raft_consensus.cc:2454] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 544627e56c3f4497a38c34cc66cf9074 in term 1.
I20260811 05:08:29.863266 472 raft_consensus.cc:2454] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 544627e56c3f4497a38c34cc66cf9074 in term 1.
I20260811 05:08:29.863371 32522 leader_election.cc:302] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c3895b16644162bf871d272a7b8ade, 544627e56c3f4497a38c34cc66cf9074; no voters:
I20260811 05:08:29.863554 564 raft_consensus.cc:2781] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:29.863765 564 raft_consensus.cc:686] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [term 1 LEADER]: Becoming Leader. State: Replica: 544627e56c3f4497a38c34cc66cf9074, State: Running, Role: LEADER
I20260811 05:08:29.864012 564 consensus_queue.cc:237] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } }
I20260811 05:08:29.865202 609 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: TxnStatusTablet state changed. Reason: New leader 544627e56c3f4497a38c34cc66cf9074. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.865332 609 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:29.865670 612 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:08:29.865870 612 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:08:29.866575 32414 catalog_manager.cc:5495] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 reported cstate change: term changed from 0 to 1, leader changed from <none> to 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:29.911326 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.907231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35791 (local address 127.0.0.1:60660)
0811 05:08:29.907501 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.907513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.907605 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.907973 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.907975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.907981 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:29.908070 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.908074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.909024 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.909026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.909604 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.909611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.910192 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.910194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.910257 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.910300 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.911078 (+ 778us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.911091 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.911147 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.911217 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 05:08:29.911372 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.907360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.30.88.193:35791)
0811 05:08:29.907508 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.907513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.907559 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:08:29.907751 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.907755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.907840 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.907958 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:29.908176 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.908917 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.909721 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.910062 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.910187 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.910441 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.910984 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.911101 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.911164 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.911221 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:08:29.914853 304 raft_consensus.cc:1261] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Refusing update from remote peer 544627e56c3f4497a38c34cc66cf9074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.915059 472 raft_consensus.cc:1261] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Refusing update from remote peer 544627e56c3f4497a38c34cc66cf9074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.915485 609 consensus_queue.cc:1035] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.915684 564 consensus_queue.cc:1035] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.916977 565 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: TxnStatusTablet state changed. Reason: New leader 544627e56c3f4497a38c34cc66cf9074. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.917078 565 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.917306 580 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: TxnStatusTablet state changed. Reason: New leader 544627e56c3f4497a38c34cc66cf9074. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.917418 580 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.919612 564 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.919716 564 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:29.919854 564 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.919970 564 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 544627e56c3f4497a38c34cc66cf9074: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:29.920077 572 mvcc.cc:205] Tried to move back new op lower bound from 7317196431004037120 to 7317196430805336064. Current Snapshot: MvccSnapshot[applied={T|T < 7317196431004037120 or (T in {7317196431004037120})}]
I20260811 05:08:29.920244 580 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.920352 580 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 564adeb87da8408baee87d42a6ffdbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.920360 573 mvcc.cc:205] Tried to move back new op lower bound from 7317196431004037120 to 7317196430805336064. Current Snapshot: MvccSnapshot[applied={T|T < 7317196431004037120}]
I20260811 05:08:29.921068 565 tablet_replica.cc:439] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "544627e56c3f4497a38c34cc66cf9074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 } } peers { permanent_uuid: "40c3895b16644162bf871d272a7b8ade" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33537 } } peers { permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 } } }
I20260811 05:08:29.921169 565 tablet_replica.cc:442] T c3568b48adae4662b15cc0534c208522 P 40c3895b16644162bf871d272a7b8ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:29.929737 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.926197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.30.88.194:33537)
0811 05:08:29.926303 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.926307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.926396 (+ 89us) server_negotiation.cc:407] Connection header received
0811 05:08:29.926573 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.926576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.926639 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.926742 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:29.927018 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.927507 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.928375 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.928610 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.928717 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.928972 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.929350 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.929482 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.929542 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.929589 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:29.940986 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.936537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:58797 (local address 127.30.88.193:35791)
0811 05:08:29.936646 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.936650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.936896 (+ 246us) server_negotiation.cc:407] Connection header received
0811 05:08:29.937043 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.937046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.937093 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.937169 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:29.937450 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.938117 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.939730 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.940073 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.940172 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.940528 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.940603 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.940678 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.940785 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.940844 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:08:29.941002 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.936407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:35791 (local address 127.30.88.194:58797)
0811 05:08:29.936806 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.936820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.936920 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.937194 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.937197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.937204 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:29.937313 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.937319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.938242 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.938246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.939608 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.939619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.940248 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.940251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.940343 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.940390 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.940692 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.940697 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.940756 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.940838 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":131,"threads_started":1}
I20260811 05:08:29.949018 472 raft_consensus.cc:1261] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Refusing update from remote peer 40c3895b16644162bf871d272a7b8ade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.949348 32604 raft_consensus.cc:1261] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Refusing update from remote peer 40c3895b16644162bf871d272a7b8ade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:29.949817 595 consensus_queue.cc:1035] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "564adeb87da8408baee87d42a6ffdbd1" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 35163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.950191 595 consensus_queue.cc:1035] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "544627e56c3f4497a38c34cc66cf9074" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 35791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:29.974268 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.970567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:57319 (local address 127.30.88.194:33537)
0811 05:08:29.970684 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:29.970689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:29.970982 (+ 293us) server_negotiation.cc:407] Connection header received
0811 05:08:29.971100 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:29.971102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:29.971141 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:29.971201 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:29.971475 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.971992 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.973224 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.973449 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.973527 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.973816 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.973903 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.973988 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:29.974086 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 05:08:29.974131 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:08:29.974308 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:29.970528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33537 (local address 127.30.88.193:57319)
0811 05:08:29.970891 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:29.970908 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:29.971014 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:29.971237 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:29.971240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:29.971256 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:29.971378 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.971385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.972148 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.972153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.973113 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:29.973124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:29.973560 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:29.973564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:29.973645 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:29.973693 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:29.973995 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:29.974005 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 05:08:29.974077 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:08:29.974148 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260811 05:08:30.001395 32414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:30.001574 32414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:30.003729 32414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed06e4a4886c40daa6a4405537dd1b44: Sending DeleteTablet for 3 replicas of tablet 5bd025c718d24c8d8e95427c5924ac88
I20260811 05:08:30.004276 32581 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60656
I20260811 05:08:30.004490 430 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:37928
I20260811 05:08:30.004693 584 tablet_replica.cc:330] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: stopping tablet replica
I20260811 05:08:30.004688 585 tablet_replica.cc:330] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: stopping tablet replica
I20260811 05:08:30.004817 584 raft_consensus.cc:2229] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:30.004805 32750 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bd025c718d24c8d8e95427c5924ac88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:29 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55438
I20260811 05:08:30.004822 585 raft_consensus.cc:2229] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:30.004930 584 raft_consensus.cc:2258] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:30.004946 585 raft_consensus.cc:2258] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:30.005125 587 tablet_replica.cc:330] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: stopping tablet replica
I20260811 05:08:30.005241 587 raft_consensus.cc:2229] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:30.005553 587 raft_consensus.cc:2258] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:30.006043 585 ts_tablet_manager.cc:1905] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:30.006623 584 ts_tablet_manager.cc:1905] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:30.009080 585 ts_tablet_manager.cc:1918] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:30.009152 585 log.cc:1196] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/5bd025c718d24c8d8e95427c5924ac88
I20260811 05:08:30.009357 584 ts_tablet_manager.cc:1918] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:30.009419 585 ts_tablet_manager.cc:1939] T 5bd025c718d24c8d8e95427c5924ac88 P 564adeb87da8408baee87d42a6ffdbd1: Deleting consensus metadata
I20260811 05:08:30.009428 584 log.cc:1196] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/5bd025c718d24c8d8e95427c5924ac88
I20260811 05:08:30.009663 584 ts_tablet_manager.cc:1939] T 5bd025c718d24c8d8e95427c5924ac88 P 544627e56c3f4497a38c34cc66cf9074: Deleting consensus metadata
I20260811 05:08:30.010119 32402 catalog_manager.cc:4841] TS 564adeb87da8408baee87d42a6ffdbd1 (127.30.88.195:35163): tablet 5bd025c718d24c8d8e95427c5924ac88 (table test-table [id=d1493ddce59f43beaa66b79565544ccc]) successfully deleted
I20260811 05:08:30.010258 32399 catalog_manager.cc:4841] TS 544627e56c3f4497a38c34cc66cf9074 (127.30.88.193:35791): tablet 5bd025c718d24c8d8e95427c5924ac88 (table test-table [id=d1493ddce59f43beaa66b79565544ccc]) successfully deleted
I20260811 05:08:30.010967 587 ts_tablet_manager.cc:1905] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:30.013019 587 ts_tablet_manager.cc:1918] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:30.013114 587 log.cc:1196] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/5bd025c718d24c8d8e95427c5924ac88
I20260811 05:08:30.013350 587 ts_tablet_manager.cc:1939] T 5bd025c718d24c8d8e95427c5924ac88 P 40c3895b16644162bf871d272a7b8ade: Deleting consensus metadata
I20260811 05:08:30.013976 32401 catalog_manager.cc:4841] TS 40c3895b16644162bf871d272a7b8ade (127.30.88.194:33537): tablet 5bd025c718d24c8d8e95427c5924ac88 (table test-table [id=d1493ddce59f43beaa66b79565544ccc]) successfully deleted
Aug 11 05:08:30 dist-test-slave-7c35 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:30.025908 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:30.015113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.30.88.193:35791)
0811 05:08:30.015226 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:30.015232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:30.015528 (+ 296us) server_negotiation.cc:407] Connection header received
0811 05:08:30.015702 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:30.015705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:30.015778 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:30.015849 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:30.016867 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:30.017356 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:30.018222 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:30.018574 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:30.018637 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:30.021940 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:30.021962 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:30.021974 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:30.021996 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:30.023915 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:30.024430 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:30.024437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:30.024439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:30.024528 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:30.024894 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:30.024898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:30.024901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:30.025162 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:30.025399 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:30.025585 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 05:08:30.025711 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 05:08:30.026327 32651 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60668
I20260811 05:08:30.041028 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:30.027971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.30.88.254:38753)
0811 05:08:30.028100 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:30.028105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:30.028582 (+ 477us) server_negotiation.cc:407] Connection header received
0811 05:08:30.028698 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:30.028700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:30.028754 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:30.028846 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:30.029484 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:30.030005 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:30.030811 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:30.031125 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:30.031232 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:30.037798 (+ 6566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:30.037801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:30.037803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:30.037826 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:30.039376 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:30.039782 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:30.039789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:30.039790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:30.039872 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:30.040144 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:30.040148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:30.040149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:30.040323 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:30.040456 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:30.040701 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 05:08:30.040821 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 05:08:30.041592 32414 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53776
I20260811 05:08:30.043205 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 32499
I20260811 05:08:30.056910 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 32671
I20260811 05:08:30.065897 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 372
I20260811 05:08:30.074335 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 32378
2026-08-11T05:08:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3071 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:30 dist-test-slave-7c35 krb5kdc[638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:30 dist-test-slave-7c35 krb5kdc[638](info): set up 2 sockets
Aug 11 05:08:30 dist-test-slave-7c35 krb5kdc[638](info): commencing operation
krb5kdc: starting...
W20260811 05:08:31.171139 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:31.291662 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46099
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:46099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:31.398713 656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:31.398986 656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:31.399044 656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:31.402549 656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:31.402596 656 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:08:31.402613 656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:31.402673 656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:31.402704 656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:31.402735 656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:31.407473 656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:46099
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:31.408298 656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:31.409162 656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:31.416980 661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:31.417021 656 server_base.cc:1034] running on GCE node
W20260811 05:08:31.416988 662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:31.416988 664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:31.417685 656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:31.418807 656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:31.422108 656 hybrid_clock.cc:648] HybridClock initialized: now 1786424911422087 us; error 31 us; skew 500 ppm
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:31.428709 656 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:31.429386 656 webserver.cc:460] Webserver started at http://127.30.88.254:45503/ using document root <none> and password file <none>
I20260811 05:08:31.429818 656 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:31.429868 656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:31.430007 656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:31.431927 656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "446d3eaa846d45818e386cdad53454dc"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "f3655585adbff35e69adc79222d0fb90"
server_key_iv: "99c5c282252a457f5f02124d3f60c674"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.432422 656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "446d3eaa846d45818e386cdad53454dc"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "f3655585adbff35e69adc79222d0fb90"
server_key_iv: "99c5c282252a457f5f02124d3f60c674"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.435761 656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:31.437899 672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:31.438992 656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:08:31.439077 656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "446d3eaa846d45818e386cdad53454dc"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "f3655585adbff35e69adc79222d0fb90"
server_key_iv: "99c5c282252a457f5f02124d3f60c674"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.439149 656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:31.457509 656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:31.460296 656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:31.460446 656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:31.467962 656 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:46099
I20260811 05:08:31.467996 749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:46099 every 8 connection(s)
I20260811 05:08:31.468822 656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:31.471531 752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:31.476537 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Bootstrap starting.
I20260811 05:08:31.477564 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 656
I20260811 05:08:31.477650 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:31.477896 31075 external_mini_cluster.cc:1428] Setting key d94f7faf8795d9744387edb808fad1ba
I20260811 05:08:31.479151 752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:31.480021 752 log.cc:824] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:31.482092 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: No bootstrap required, opened a new log
I20260811 05:08:31.486109 752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } }
I20260811 05:08:31.486299 752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:31.486332 752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446d3eaa846d45818e386cdad53454dc, State: Initialized, Role: FOLLOWER
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:31.486814 752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } }
I20260811 05:08:31.487001 752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:31.487088 752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:31.487191 752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:31.488386 752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } }
I20260811 05:08:31.488725 752 leader_election.cc:302] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 446d3eaa846d45818e386cdad53454dc; no voters:
I20260811 05:08:31.489064 752 leader_election.cc:288] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:31.489142 758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:31.489353 758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [term 1 LEADER]: Becoming Leader. State: Replica: 446d3eaa846d45818e386cdad53454dc, State: Running, Role: LEADER
I20260811 05:08:31.489629 758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } }
I20260811 05:08:31.490321 752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:31.490787 760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "446d3eaa846d45818e386cdad53454dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } } }
I20260811 05:08:31.491199 760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:31.491716 759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 446d3eaa846d45818e386cdad53454dc. Latest consensus state: current_term: 1 leader_uuid: "446d3eaa846d45818e386cdad53454dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446d3eaa846d45818e386cdad53454dc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46099 } } }
I20260811 05:08:31.491828 759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:31.493990 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.479026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.30.88.254:46099)
0811 05:08:31.479509 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:31.479527 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:31.479566 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:08:31.480396 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:31.480418 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:31.480748 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:31.481110 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:31.481764 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.483233 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.484059 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.484753 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.484898 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.487479 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:31.487487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:31.487502 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:31.487531 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:31.489871 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.490429 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.490437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.490444 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.490591 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.491018 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.491024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.491027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.491576 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:31.492461 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:31.492653 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 05:08:31.492991 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":176,"threads_started":1}
W20260811 05:08:31.495653 774 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:08:31.495731 774 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:08:31.495975 764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:31.496774 764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:31.502170 764 catalog_manager.cc:1347] Generated new cluster ID: 24926f74138049c9a49d1f66cbbdd097
I20260811 05:08:31.502234 764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:31.509620 764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:31.510741 764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:31.516739 764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Generated new TSK 0
I20260811 05:08:31.517416 764 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:31.555241 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46099
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:31.665940 780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:31.666185 780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:31.666229 780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:31.670195 780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:31.670249 780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:31.670320 780 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:08:31.670353 780 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:31.670444 780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:31.675438 780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:31.676347 780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:31.677304 780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:31.686596 786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:31.686669 787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:31.686599 789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:31.686718 780 server_base.cc:1034] running on GCE node
I20260811 05:08:31.687399 780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:31.688012 780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:31.689170 780 hybrid_clock.cc:648] HybridClock initialized: now 1786424911689160 us; error 24 us; skew 500 ppm
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:31.692276 780 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:31.693348 780 webserver.cc:460] Webserver started at http://127.30.88.193:45487/ using document root <none> and password file <none>
I20260811 05:08:31.693917 780 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:31.693993 780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:31.694166 780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:31.696159 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "6c62631d7fee4d8caef70ef019f83b0c"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "63c08f9232c3f6984da1c55dca4a4401"
server_key_iv: "7840def96bde23e8876c261284edd546"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.696681 780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "6c62631d7fee4d8caef70ef019f83b0c"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "63c08f9232c3f6984da1c55dca4a4401"
server_key_iv: "7840def96bde23e8876c261284edd546"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.700078 780 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:31.702207 799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:31.703423 780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:31.703573 780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "6c62631d7fee4d8caef70ef019f83b0c"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "63c08f9232c3f6984da1c55dca4a4401"
server_key_iv: "7840def96bde23e8876c261284edd546"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.703679 780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:31.756201 780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:31.759541 780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:31.759754 780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:31.761168 780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:31.761240 780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:31.761310 780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:31.761358 780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:31.772228 780 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:34605
I20260811 05:08:31.772328 935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:34605 every 8 connection(s)
I20260811 05:08:31.773149 780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:31.775111 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.762271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:46967 (local address 127.30.88.254:46099)
0811 05:08:31.762431 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:31.762437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:31.763281 (+ 844us) server_negotiation.cc:407] Connection header received
0811 05:08:31.764356 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:31.764360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:31.764417 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:31.764537 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:31.766060 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.766993 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.768352 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.768785 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.768901 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.771359 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:31.771363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:31.771368 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:31.771415 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:31.772949 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.773539 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.773546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.773549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.773666 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.774052 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.774055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.774057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.774281 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:31.774400 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:31.774803 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 05:08:31.774974 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:08:31.775856 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.762638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.193:46967)
0811 05:08:31.763112 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:31.763161 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:31.764102 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:31.764708 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:31.764718 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:31.765219 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:31.765875 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.765896 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.767135 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.767139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.768216 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.768225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.768965 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.768970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.769152 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.769686 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:31.769701 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:31.771159 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:31.773073 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.773078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.773092 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.773435 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.773782 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.773785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.773787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.773944 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.774383 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:31.774387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:31.774591 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 05:08:31.774740 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:08:31.775033 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":164,"threads_started":1}
I20260811 05:08:31.777256 936 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46099
I20260811 05:08:31.777405 936 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:31.777890 936 heartbeater.cc:511] Master 127.30.88.254:46099 requested a full tablet report, sending...
I20260811 05:08:31.779532 690 ts_manager.cc:194] Registered new tserver with Master: 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605)
I20260811 05:08:31.780828 690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:46967
I20260811 05:08:31.781612 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 780
I20260811 05:08:31.781744 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:31.782042 31075 external_mini_cluster.cc:1428] Setting key 49eaa5b818e9dcb2678bef77e0606e2b
I20260811 05:08:31.786652 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.779383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:54731 (local address 127.30.88.254:46099)
0811 05:08:31.779502 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:31.779506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:31.779674 (+ 168us) server_negotiation.cc:407] Connection header received
0811 05:08:31.779775 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:31.779778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:31.779820 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:31.779887 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:31.780341 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.780825 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.781874 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.782087 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.782189 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.783781 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:31.783790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:31.783792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:31.783812 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:31.785099 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.785515 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.785518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.785520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.785594 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.785866 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.785869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.785871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.786028 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:31.786136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:31.786397 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:08:31.786493 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":50}
I20260811 05:08:31.786651 943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.779267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.193:54731)
0811 05:08:31.779565 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:31.779580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:31.779691 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:31.779936 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:31.779939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:31.780126 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:31.780232 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.780238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.780972 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.780975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.781759 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.781768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.782202 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.782205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.782312 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.783207 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:31.783245 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:31.783647 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:31.785208 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.785213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.785215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.785413 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.785685 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.785689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.785691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.785776 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.786125 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:31.786128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:31.786221 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 05:08:31.786326 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 05:08:31.786458 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:31.809564 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46099
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:31.917811 946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:31.918064 946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:31.918141 946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:31.922065 946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:31.922144 946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:31.922212 946 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:08:31.922262 946 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:31.922348 946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:31.927215 946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:31.928210 946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:31.929143 946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:31.938277 957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:31.938285 954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:31.938441 946 server_base.cc:1034] running on GCE node
W20260811 05:08:31.938285 955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:31.938977 946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:31.939527 946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:31.940737 946 hybrid_clock.cc:648] HybridClock initialized: now 1786424911940721 us; error 35 us; skew 500 ppm
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:31.943600 946 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:31.944378 946 webserver.cc:460] Webserver started at http://127.30.88.194:45167/ using document root <none> and password file <none>
I20260811 05:08:31.944926 946 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:31.945024 946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:31.945250 946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:31.947211 946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "1aff25f1226e4f80b97b748978954704"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "6a2d3fb1736facbbd553da7c9be52f16"
server_key_iv: "5e651c540a56c324977b27980c61e3e2"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.947768 946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "1aff25f1226e4f80b97b748978954704"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "6a2d3fb1736facbbd553da7c9be52f16"
server_key_iv: "5e651c540a56c324977b27980c61e3e2"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.951460 946 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:31.953884 965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:31.955101 946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:31.955238 946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "1aff25f1226e4f80b97b748978954704"
format_stamp: "Formatted at 2026-08-11 05:08:31 on dist-test-slave-7c35"
server_key: "6a2d3fb1736facbbd553da7c9be52f16"
server_key_iv: "5e651c540a56c324977b27980c61e3e2"
server_key_version: "encryptionkey@0"
I20260811 05:08:31.955346 946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:31.979439 946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:31.982393 946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:31.982599 946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:31.984012 946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:31.984094 946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:31.984153 946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:31.984210 946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:31 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:31.995512 946 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:42609
I20260811 05:08:31.995599 1106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:42609 every 8 connection(s)
I20260811 05:08:31.996484 946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:31.998039 977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.985304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.194:43015)
0811 05:08:31.985747 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:31.985785 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:31.986705 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:31.987453 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:31.987458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:31.987761 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:31.988178 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.988200 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.989002 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.989005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.989836 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:31.989845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.990418 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.990423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:31.990665 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.991297 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:31.991313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:31.993624 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:31.995571 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.995578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.995591 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.995903 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.996193 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:31.996195 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:31.996196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:31.996311 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:31.996706 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:31.996710 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:31.996953 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 05:08:31.997129 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 05:08:31.997405 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
I20260811 05:08:31.998579 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:31.984820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:43015 (local address 127.30.88.254:46099)
0811 05:08:31.984936 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:31.984940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:31.985922 (+ 982us) server_negotiation.cc:407] Connection header received
0811 05:08:31.986961 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:31.986966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:31.987028 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:31.987085 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:31.988328 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.988862 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.989960 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:31.990258 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:31.990389 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:31.993784 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:31.993788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:31.993790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:31.993820 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:31.995474 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.996006 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.996009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.996011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.996100 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:31.996402 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:31.996406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:31.996407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:31.996614 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:31.996707 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:31.998385 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0811 05:08:31.998481 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:08:31.999554 1107 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46099
I20260811 05:08:31.999689 1107 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:32.000185 1107 heartbeater.cc:511] Master 127.30.88.254:46099 requested a full tablet report, sending...
I20260811 05:08:32.001168 688 ts_manager.cc:194] Registered new tserver with Master: 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609)
I20260811 05:08:32.001833 688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:43015
I20260811 05:08:32.005517 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 946
I20260811 05:08:32.005641 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:32.005966 31075 external_mini_cluster.cc:1428] Setting key 4007159b59458691ff79f056b1cf053c
I20260811 05:08:32.008595 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.001394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.194:38681)
0811 05:08:32.002133 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.002143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.002263 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.002545 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.002548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.002753 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:32.002880 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.002889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.003603 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.003606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.004191 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.004200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.004617 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.004620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.004687 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.005046 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:32.005056 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:32.005328 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:32.007171 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.007174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.007175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.007366 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.007664 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.007665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.007667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.007765 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.008225 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:32.008227 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:32.008304 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.008372 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.008477 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":98,"threads_started":1}
I20260811 05:08:32.008796 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.001467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:38681 (local address 127.30.88.254:46099)
0811 05:08:32.001565 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.001568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.002220 (+ 652us) server_negotiation.cc:407] Connection header received
0811 05:08:32.002388 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.002404 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.002442 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.002503 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:32.002992 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.003462 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.004293 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.004522 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.004600 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.005476 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:32.005481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:32.005484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:32.005510 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:32.007041 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.007487 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.007490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.007492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.007578 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.007864 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.007869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.007873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.008132 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:32.008234 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.008471 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.008593 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:32.033442 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46099
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:32.140640 1116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:32.140892 1116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:32.140973 1116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:32.144981 1116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:32.145052 1116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:32.145123 1116 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:08:32.145164 1116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:32.145231 1116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:32.150240 1116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:32.151196 1116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:32.152140 1116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:32.161474 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:32.161499 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:32.161527 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:32.161693 1116 server_base.cc:1034] running on GCE node
I20260811 05:08:32.162114 1116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:32.162688 1116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:32.163870 1116 hybrid_clock.cc:648] HybridClock initialized: now 1786424912163861 us; error 44 us; skew 500 ppm
Aug 11 05:08:32 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424912, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:32.166386 1116 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:32.167229 1116 webserver.cc:460] Webserver started at http://127.30.88.195:40781/ using document root <none> and password file <none>
I20260811 05:08:32.167749 1116 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:32.167836 1116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:32.168058 1116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:32.169976 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
format_stamp: "Formatted at 2026-08-11 05:08:32 on dist-test-slave-7c35"
server_key: "8350de2d5503fd7e0143f2ce50bf22f0"
server_key_iv: "62ecbbbb75d9db9be11cdd401c6917b3"
server_key_version: "encryptionkey@0"
I20260811 05:08:32.170538 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
format_stamp: "Formatted at 2026-08-11 05:08:32 on dist-test-slave-7c35"
server_key: "8350de2d5503fd7e0143f2ce50bf22f0"
server_key_iv: "62ecbbbb75d9db9be11cdd401c6917b3"
server_key_version: "encryptionkey@0"
I20260811 05:08:32.174082 1116 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:32.176412 1134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:32.177631 1116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:08:32.177755 1116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
format_stamp: "Formatted at 2026-08-11 05:08:32 on dist-test-slave-7c35"
server_key: "8350de2d5503fd7e0143f2ce50bf22f0"
server_key_iv: "62ecbbbb75d9db9be11cdd401c6917b3"
server_key_version: "encryptionkey@0"
I20260811 05:08:32.177852 1116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:32.192442 1116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:32.194962 1116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:32.195133 1116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:32.196362 1116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:32.196406 1116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:32.196462 1116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:32.196501 1116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:32 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424912, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:32.207139 1116 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:44083
I20260811 05:08:32.207249 1280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:44083 every 8 connection(s)
I20260811 05:08:32.208004 1116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:32.209267 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 1116
I20260811 05:08:32.209388 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:32.209748 31075 external_mini_cluster.cc:1428] Setting key a97af4077f29d7542b69d8e47a9508da
I20260811 05:08:32.210784 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.197129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:38841 (local address 127.30.88.254:46099)
0811 05:08:32.197291 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.197297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.197998 (+ 701us) server_negotiation.cc:407] Connection header received
0811 05:08:32.199174 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.199178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.199241 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.199306 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:32.200909 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.201767 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.203682 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.204033 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.204117 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.206947 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:32.206950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:32.206953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:32.206998 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:32.208618 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.209162 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.209166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.209168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.209262 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.209650 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.209654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.209655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.209885 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:32.210006 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.210503 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.210622 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:08:32.211671 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.197431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.195:38841)
0811 05:08:32.197848 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.197892 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.198913 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.199751 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.199760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.200141 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:32.200749 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.200766 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.201945 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.201949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.203533 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.203550 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.204188 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.204194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.204403 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.204997 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:32.205026 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:32.206723 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:32.208773 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.208778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.208789 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.209067 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.209373 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.209376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.209378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.209556 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.209990 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:32.209994 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:32.210250 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.210449 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.210747 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260811 05:08:32.212833 1283 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46099
I20260811 05:08:32.213016 1283 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:32.213456 1283 heartbeater.cc:511] Master 127.30.88.254:46099 requested a full tablet report, sending...
I20260811 05:08:32.214382 690 ts_manager.cc:194] Registered new tserver with Master: bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:32.215018 690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:38841
I20260811 05:08:32.220952 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.214971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.30.88.195:42277)
0811 05:08:32.215286 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.215300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.215380 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.215622 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.215624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.215812 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:32.215961 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.215970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.216633 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.216636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.217245 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.217271 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.217636 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.217638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.217703 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.218172 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:32.218183 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:32.218447 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:32.219841 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.219843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.219844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.220015 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.220252 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.220254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.220255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.220325 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.220608 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:32.220609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:32.220677 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.220751 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.220841 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260811 05:08:32.220990 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.215004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:42277 (local address 127.30.88.254:46099)
0811 05:08:32.215119 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.215122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.215366 (+ 244us) server_negotiation.cc:407] Connection header received
0811 05:08:32.215462 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.215464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.215504 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.215586 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:32.216069 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.216510 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.217358 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.217548 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.217615 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.218558 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:32.218561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:32.218563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:32.218578 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:32.219754 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.220109 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.220111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.220112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.220177 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.220394 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.220396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.220397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.220533 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:32.220601 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.220804 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.220883 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:08:32.223457 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:08:32 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:08:32 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:32.248860 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.241314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.30.88.254:46099)
0811 05:08:32.241429 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.241432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.241443 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:08:32.241480 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.241482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.241516 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.241583 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:32.242212 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.242784 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.243643 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.243883 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.243957 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.246157 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:32.246159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:32.246161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:32.246183 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:32.247574 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.247913 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.247915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.247917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.248003 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.248204 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.248206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.248207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.248344 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:32.248433 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.248587 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.248664 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:08:32.251416 690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:32.254496 690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:32.268730 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.262604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.0.0.1:59660)
0811 05:08:32.262982 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.263037 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.263155 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.264419 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.264427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.264455 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:32.264585 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.264594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.266011 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.266014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.267143 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.267153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.268120 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.268291 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.268330 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.268445 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.268574 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":51,"thread_start_us":83,"threads_started":1}
I20260811 05:08:32.268759 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.263134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34605 (local address 127.0.0.1:60670)
0811 05:08:32.263401 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.263415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.263527 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.264837 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.264842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.264860 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:32.264941 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.264947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.266229 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.266232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.267299 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.267310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.268167 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.268518 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.268538 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.268602 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.268653 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":263,"thread_start_us":88,"threads_started":1}
I20260811 05:08:32.268754 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.263574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.30.88.195:44083)
0811 05:08:32.263923 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.263937 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.263976 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:08:32.264051 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.264056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.264263 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.264424 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:32.264710 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.265833 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.267263 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.267984 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268090 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.268241 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.268499 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.268594 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
I20260811 05:08:32.269366 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.262807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:42609 (local address 127.0.0.1:56840)
0811 05:08:32.263285 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.263300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.263390 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.264633 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.264638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.264658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:32.264743 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.264747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.265763 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.265769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.267544 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.267553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.268763 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.268861 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.269112 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.269189 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.269251 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":51,"thread_start_us":164,"threads_started":1}
I20260811 05:08:32.269460 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.263598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.30.88.194:42609)
0811 05:08:32.264042 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.264055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.264077 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:08:32.264152 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.264156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.264353 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.264500 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:32.264877 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.265655 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.267668 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.268451 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268551 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.268698 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.269239 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.269318 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":95,"threads_started":1}
I20260811 05:08:32.269389 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.264103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.30.88.193:34605)
0811 05:08:32.264411 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.264426 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.264445 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:08:32.264515 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.264519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.264697 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.264848 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:32.265035 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.266100 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.267423 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.268023 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.268276 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.268430 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.269150 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.269264 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260811 05:08:32.271935 858 tablet_service.cc:1467] Processing CreateTablet for tablet 6273699876744687a1697f1cc1b14935 (DEFAULT_TABLE table=test-table [id=2dcf9fa9dba1442cb327fb79f050d138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:32.271883 1202 tablet_service.cc:1467] Processing CreateTablet for tablet 6273699876744687a1697f1cc1b14935 (DEFAULT_TABLE table=test-table [id=2dcf9fa9dba1442cb327fb79f050d138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:32.272114 1027 tablet_service.cc:1467] Processing CreateTablet for tablet 6273699876744687a1697f1cc1b14935 (DEFAULT_TABLE table=test-table [id=2dcf9fa9dba1442cb327fb79f050d138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:32.272903 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6273699876744687a1697f1cc1b14935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.272940 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6273699876744687a1697f1cc1b14935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.272933 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6273699876744687a1697f1cc1b14935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.279771 1314 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Bootstrap starting.
I20260811 05:08:32.280871 1317 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Bootstrap starting.
I20260811 05:08:32.281809 1314 tablet_bootstrap.cc:654] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.282595 1314 log.cc:824] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:32.282622 1317 tablet_bootstrap.cc:654] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.283142 1316 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Bootstrap starting.
I20260811 05:08:32.283485 1317 log.cc:824] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:32.284729 1314 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: No bootstrap required, opened a new log
I20260811 05:08:32.284947 1314 ts_tablet_manager.cc:1397] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:08:32.285066 1316 tablet_bootstrap.cc:654] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.285275 1317 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: No bootstrap required, opened a new log
I20260811 05:08:32.285454 1317 ts_tablet_manager.cc:1397] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:08:32.285827 1316 log.cc:824] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:32.287689 1316 tablet_bootstrap.cc:492] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: No bootstrap required, opened a new log
I20260811 05:08:32.287864 1316 ts_tablet_manager.cc:1397] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:08:32.288558 1317 raft_consensus.cc:348] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.288552 1314 raft_consensus.cc:348] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.288775 1317 raft_consensus.cc:374] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.288789 1314 raft_consensus.cc:374] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.288831 1314 raft_consensus.cc:729] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.288836 1317 raft_consensus.cc:729] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.289278 1314 consensus_queue.cc:260] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.289278 1317 consensus_queue.cc:260] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.290182 1317 ts_tablet_manager.cc:1428] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:08:32.290258 1314 ts_tablet_manager.cc:1428] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:08:32.290544 1107 heartbeater.cc:503] Master 127.30.88.254:46099 was elected leader, sending a full tablet report...
I20260811 05:08:32.290573 936 heartbeater.cc:503] Master 127.30.88.254:46099 was elected leader, sending a full tablet report...
I20260811 05:08:32.292428 1316 raft_consensus.cc:348] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.292706 1316 raft_consensus.cc:374] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.292774 1316 raft_consensus.cc:729] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.293170 1316 consensus_queue.cc:260] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.293790 1283 heartbeater.cc:503] Master 127.30.88.254:46099 was elected leader, sending a full tablet report...
I20260811 05:08:32.294072 1316 ts_tablet_manager.cc:1428] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 05:08:32.459095 1285 tablet.cc:2369] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:08:32.497637 1108 tablet.cc:2369] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:08:32.524406 937 tablet.cc:2369] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:32.612267 1327 raft_consensus.cc:482] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:32.612557 1327 raft_consensus.cc:504] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.613813 1327 leader_election.cc:288] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605)
I20260811 05:08:32.617748 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.614113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:42609 (local address 127.30.88.195:59901)
0811 05:08:32.614303 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.614316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.614442 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.614766 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.614770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.614790 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:32.614976 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.614989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.615765 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.615767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.616736 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.616747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.617409 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.617412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.617500 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.617521 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.617580 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.617632 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 05:08:32.617828 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.614156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59901 (local address 127.30.88.194:42609)
0811 05:08:32.614292 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.614299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.614391 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:08:32.614573 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.614576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.614631 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.614727 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:32.615121 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.615646 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.616863 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.617256 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.617398 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.617452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.617618 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.617707 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:08:32.618443 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.614407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:49657 (local address 127.30.88.193:34605)
0811 05:08:32.614578 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.614582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.614747 (+ 165us) server_negotiation.cc:407] Connection header received
0811 05:08:32.614841 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.614862 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.614916 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.615040 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:32.615758 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.616244 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.617368 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.617922 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.618046 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.618108 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.618288 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.618356 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:08:32.618451 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.614181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34605 (local address 127.30.88.195:49657)
0811 05:08:32.614652 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.614668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.614759 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.615070 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.615076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.615099 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:32.615613 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.615621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.616360 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.616363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.617241 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.617249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.618072 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.618078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.618165 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.618184 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.618244 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.618315 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":145,"threads_started":1}
I20260811 05:08:32.618611 1053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6273699876744687a1697f1cc1b14935" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704" is_pre_election: true
I20260811 05:08:32.618939 1053 raft_consensus.cc:2454] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 0.
I20260811 05:08:32.619385 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6273699876744687a1697f1cc1b14935" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c" is_pre_election: true
I20260811 05:08:32.619408 1138 leader_election.cc:302] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:32.619680 1327 raft_consensus.cc:2781] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:32.619673 881 raft_consensus.cc:2454] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 0.
I20260811 05:08:32.619769 1327 raft_consensus.cc:482] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:32.619817 1327 raft_consensus.cc:3037] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:32.620986 1327 raft_consensus.cc:504] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.621316 1327 leader_election.cc:288] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 election: Requested vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605)
I20260811 05:08:32.621635 1053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6273699876744687a1697f1cc1b14935" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704"
I20260811 05:08:32.621686 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6273699876744687a1697f1cc1b14935" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c"
I20260811 05:08:32.621776 881 raft_consensus.cc:3037] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:32.621776 1053 raft_consensus.cc:3037] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:32.623021 881 raft_consensus.cc:2454] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 1.
I20260811 05:08:32.623023 1053 raft_consensus.cc:2454] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 1.
I20260811 05:08:32.623348 1138 leader_election.cc:302] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, 6c62631d7fee4d8caef70ef019f83b0c, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:32.623553 1327 raft_consensus.cc:2781] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:32.623728 1327 raft_consensus.cc:686] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 LEADER]: Becoming Leader. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Running, Role: LEADER
I20260811 05:08:32.624051 1327 consensus_queue.cc:237] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } }
I20260811 05:08:32.626892 688 catalog_manager.cc:5495] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d reported cstate change: term changed from 0 to 1, leader changed from <none> to bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195). New cstate: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } health_report { overall_health: UNKNOWN } } }
Aug 11 05:08:32 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:32.739898 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.732332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46099 (local address 127.0.0.1:55522)
0811 05:08:32.732631 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:32.732643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:32.732739 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:32.732955 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:32.732957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:32.733318 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:32.733412 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.733417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.734109 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.734111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.734643 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:32.734652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.735102 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.735105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:32.735178 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.735777 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:32.735789 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:32.737108 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:32.738479 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.738487 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.738491 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.738692 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.738967 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:32.738969 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:32.738970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:32.739043 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:32.739402 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:32.739407 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:32.739498 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 05:08:32.739585 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:08:32.739721 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":102,"threads_started":1}
I20260811 05:08:32.739946 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:32.732457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.30.88.254:46099)
0811 05:08:32.732611 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:32.732618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:32.732713 (+ 95us) server_negotiation.cc:407] Connection header received
0811 05:08:32.732804 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:32.732806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:32.732846 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:32.732913 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:32.733511 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.734016 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.734750 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:32.734994 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:32.735082 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:32.737238 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:32.737240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:32.737242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:32.737261 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:32.738387 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.738770 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.738772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.738773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.738901 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:32.739118 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:32.739120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:32.739121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:32.739272 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:32.739401 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:32.739642 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 05:08:32.739776 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":75}
I20260811 05:08:32.743697 688 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55522:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:08:32.744423 688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:32.749029 858 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec6eba2d58466e852c8e2a40aeff49 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e294a9e41fab48d082c427851661a9f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:32.749151 1027 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec6eba2d58466e852c8e2a40aeff49 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e294a9e41fab48d082c427851661a9f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:32.749737 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec6eba2d58466e852c8e2a40aeff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.749737 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec6eba2d58466e852c8e2a40aeff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.749315 1202 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec6eba2d58466e852c8e2a40aeff49 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e294a9e41fab48d082c427851661a9f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:08:32.749989 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec6eba2d58466e852c8e2a40aeff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:32.752729 1317 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: Bootstrap starting.
I20260811 05:08:32.752729 1314 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: Bootstrap starting.
I20260811 05:08:32.752736 1316 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: Bootstrap starting.
I20260811 05:08:32.753545 1316 tablet_bootstrap.cc:654] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.753685 1314 tablet_bootstrap.cc:654] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.753685 1317 tablet_bootstrap.cc:654] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:32.754948 1317 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: No bootstrap required, opened a new log
I20260811 05:08:32.755102 1317 ts_tablet_manager.cc:1397] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:32.755221 1314 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: No bootstrap required, opened a new log
I20260811 05:08:32.755323 1314 ts_tablet_manager.cc:1397] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:32.755549 1316 tablet_bootstrap.cc:492] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: No bootstrap required, opened a new log
I20260811 05:08:32.755649 1316 ts_tablet_manager.cc:1397] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:08:32.755622 1317 raft_consensus.cc:348] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.755740 1317 raft_consensus.cc:374] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.755780 1317 raft_consensus.cc:729] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.755722 1314 raft_consensus.cc:348] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.755823 1314 raft_consensus.cc:374] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.755861 1314 raft_consensus.cc:729] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.755910 1317 consensus_queue.cc:260] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.755985 1314 consensus_queue.cc:260] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.756103 1316 raft_consensus.cc:348] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.756165 1317 ts_tablet_manager.cc:1428] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:32.756225 1314 ts_tablet_manager.cc:1428] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:08:32.756230 1316 raft_consensus.cc:374] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:32.756268 1316 raft_consensus.cc:729] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Initialized, Role: FOLLOWER
I20260811 05:08:32.756385 1316 consensus_queue.cc:260] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:32.756664 1316 ts_tablet_manager.cc:1428] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:32.756672 1327 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:32.756834 1327 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:32.756878 1325 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:32.757015 1325 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:32.757211 1324 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:32.757364 1324 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:32.999424 1327 raft_consensus.cc:482] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:32.999620 1327 raft_consensus.cc:504] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.000142 1327 leader_election.cc:288] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605), 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609)
I20260811 05:08:33.000586 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec6eba2d58466e852c8e2a40aeff49" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c" is_pre_election: true
I20260811 05:08:33.000593 1053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec6eba2d58466e852c8e2a40aeff49" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704" is_pre_election: true
I20260811 05:08:33.000739 1053 raft_consensus.cc:2454] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 0.
I20260811 05:08:33.000751 881 raft_consensus.cc:2454] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 0.
I20260811 05:08:33.001037 1138 leader_election.cc:302] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:33.001202 1327 raft_consensus.cc:2781] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:33.001291 1327 raft_consensus.cc:482] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:33.001343 1327 raft_consensus.cc:3037] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.002279 1327 raft_consensus.cc:504] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.002665 1327 leader_election.cc:288] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 election: Requested vote from peers 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605), 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609)
I20260811 05:08:33.003046 1053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec6eba2d58466e852c8e2a40aeff49" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704"
I20260811 05:08:33.003078 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec6eba2d58466e852c8e2a40aeff49" candidate_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c"
I20260811 05:08:33.003185 1053 raft_consensus.cc:3037] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.003188 881 raft_consensus.cc:3037] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.004132 1053 raft_consensus.cc:2454] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 1.
I20260811 05:08:33.004312 881 raft_consensus.cc:2454] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc0d7fae98d4449adf317bfbed3c37d in term 1.
I20260811 05:08:33.004413 1138 leader_election.cc:302] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:33.004585 1327 raft_consensus.cc:2781] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:33.004679 1327 raft_consensus.cc:686] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 LEADER]: Becoming Leader. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Running, Role: LEADER
I20260811 05:08:33.004820 1327 consensus_queue.cc:237] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.005460 1330 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: TxnStatusTablet state changed. Reason: New leader bfc0d7fae98d4449adf317bfbed3c37d. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.005579 1330 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:33.005879 1345 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:08:33.006114 1345 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:08:33.006541 686 catalog_manager.cc:5495] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d reported cstate change: term changed from 0 to 1, leader changed from <none> to bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195). New cstate: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } health_report { overall_health: HEALTHY } } }
I20260811 05:08:33.036767 1330 consensus_queue.cc:1035] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.045691 1327 consensus_queue.cc:1035] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.051741 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.046495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.30.88.195:44083)
0811 05:08:33.047176 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.047190 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.047216 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:33.047278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.047281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.047352 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.047805 (+ 453us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.047913 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.048703 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.049929 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.050321 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.050438 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.050778 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.051335 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.051423 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.051549 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.051605 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":87,"threads_started":1}
I20260811 05:08:33.051779 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.046582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.0.0.1:59674)
0811 05:08:33.046772 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.046788 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.046916 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.047513 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.047516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.047524 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:33.047681 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.047690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.048820 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.048835 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.049802 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.049813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.050447 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.050451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.050543 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.050596 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.051413 (+ 817us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.051430 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.051538 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.051630 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 05:08:33.056221 1053 raft_consensus.cc:1261] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Refusing update from remote peer bfc0d7fae98d4449adf317bfbed3c37d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:33.056245 881 raft_consensus.cc:1261] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Refusing update from remote peer bfc0d7fae98d4449adf317bfbed3c37d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:33.056633 1353 consensus_queue.cc:1035] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.056778 1327 consensus_queue.cc:1035] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.059324 1324 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: TxnStatusTablet state changed. Reason: New leader bfc0d7fae98d4449adf317bfbed3c37d. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.059438 1324 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:33.060679 1325 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: TxnStatusTablet state changed. Reason: New leader bfc0d7fae98d4449adf317bfbed3c37d. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.060788 1325 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:33.061897 1353 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.061998 1353 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:33.062614 1325 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.062716 1325 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 1aff25f1226e4f80b97b748978954704: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:33.064495 1330 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.064605 1330 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P bfc0d7fae98d4449adf317bfbed3c37d: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:08:33.065768 1324 tablet_replica.cc:439] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } } }
I20260811 05:08:33.065871 1324 tablet_replica.cc:442] T 44ec6eba2d58466e852c8e2a40aeff49 P 6c62631d7fee4d8caef70ef019f83b0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:08:33.067020 1351 mvcc.cc:205] Tried to move back new op lower bound from 7317196443873767424 to 7317196443668922368. Current Snapshot: MvccSnapshot[applied={T|T < 7317196443873767424 or (T in {7317196443873767424})}]
I20260811 05:08:33.074229 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.070912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.30.88.195:44083)
0811 05:08:33.071053 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.071059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.071260 (+ 201us) server_negotiation.cc:407] Connection header received
0811 05:08:33.071419 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.071424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.071471 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.071547 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.071819 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.072331 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.073206 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.073424 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.073531 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.073803 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.073879 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.073966 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.074055 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.074100 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:08:33.086702 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.083164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:36455 (local address 127.30.88.195:44083)
0811 05:08:33.083275 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.083278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.083562 (+ 284us) server_negotiation.cc:407] Connection header received
0811 05:08:33.083681 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.083683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.083718 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.083780 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.084044 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.084477 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.085718 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.085923 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.085999 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.086321 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.086377 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.086431 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.086552 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.086590 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:33.086838 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.083081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.30.88.195:36455)
0811 05:08:33.083476 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.083490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.083585 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.083812 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.083815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.083822 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:33.083945 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.083951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.084605 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.084609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.085597 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.085609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.086026 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.086030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.086128 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.086176 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.086458 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.086465 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.086530 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.086601 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":110,"threads_started":1}
I20260811 05:08:33.128060 687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:33.128247 687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:33.130589 687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Sending DeleteTablet for 3 replicas of tablet 6273699876744687a1697f1cc1b14935
I20260811 05:08:33.131428 1027 tablet_service.cc:1514] Processing DeleteTablet for tablet 6273699876744687a1697f1cc1b14935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:33 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:56840
I20260811 05:08:33.131493 1202 tablet_service.cc:1514] Processing DeleteTablet for tablet 6273699876744687a1697f1cc1b14935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:33 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:59660
I20260811 05:08:33.131857 858 tablet_service.cc:1514] Processing DeleteTablet for tablet 6273699876744687a1697f1cc1b14935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:33 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60670
I20260811 05:08:33.132112 687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:08:33.132215 1373 tablet_replica.cc:330] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: stopping tablet replica
I20260811 05:08:33.132273 1372 tablet_replica.cc:330] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: stopping tablet replica
W20260811 05:08:33.132547 687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:33.132606 1372 raft_consensus.cc:2229] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:33.132761 1373 raft_consensus.cc:2229] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:33.132812 1372 raft_consensus.cc:2258] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:33.133154 1373 raft_consensus.cc:2258] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:33.134172 1373 ts_tablet_manager.cc:1905] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:33.134172 1372 ts_tablet_manager.cc:1905] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:33.137178 1373 ts_tablet_manager.cc:1918] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:33.137367 1373 log.cc:1196] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/6273699876744687a1697f1cc1b14935
I20260811 05:08:33.137562 858 tablet_service.cc:1467] Processing CreateTablet for tablet 9526b63c12ae483b894217708d5bd59e (DEFAULT_TABLE table=test-table [id=fb9b8683d92c4499b880bda29ad8604c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:33.137872 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9526b63c12ae483b894217708d5bd59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:33.138017 1202 tablet_service.cc:1467] Processing CreateTablet for tablet 9526b63c12ae483b894217708d5bd59e (DEFAULT_TABLE table=test-table [id=fb9b8683d92c4499b880bda29ad8604c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:33.138067 1027 tablet_service.cc:1467] Processing CreateTablet for tablet 9526b63c12ae483b894217708d5bd59e (DEFAULT_TABLE table=test-table [id=fb9b8683d92c4499b880bda29ad8604c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:33.138237 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9526b63c12ae483b894217708d5bd59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:33.140314 1314 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Bootstrap starting.
I20260811 05:08:33.140511 1373 ts_tablet_manager.cc:1939] T 6273699876744687a1697f1cc1b14935 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting consensus metadata
I20260811 05:08:33.140640 1372 ts_tablet_manager.cc:1918] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:33.140733 1372 log.cc:1196] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/6273699876744687a1697f1cc1b14935
I20260811 05:08:33.141057 1372 ts_tablet_manager.cc:1939] T 6273699876744687a1697f1cc1b14935 P 1aff25f1226e4f80b97b748978954704: Deleting consensus metadata
I20260811 05:08:33.141264 1314 tablet_bootstrap.cc:654] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:33.141921 1375 tablet_replica.cc:330] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: stopping tablet replica
I20260811 05:08:33.142002 1316 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Bootstrap starting.
I20260811 05:08:33.142220 1375 raft_consensus.cc:2229] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:33.142447 1375 raft_consensus.cc:2258] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:33.142971 1316 tablet_bootstrap.cc:654] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:33.143121 676 catalog_manager.cc:4841] TS bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083): tablet 6273699876744687a1697f1cc1b14935 (table test-table [id=2dcf9fa9dba1442cb327fb79f050d138]) successfully deleted
I20260811 05:08:33.143432 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9526b63c12ae483b894217708d5bd59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:33.143769 1375 ts_tablet_manager.cc:1905] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:33.144134 1316 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: No bootstrap required, opened a new log
I20260811 05:08:33.144224 1316 ts_tablet_manager.cc:1397] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:08:33.144665 1316 raft_consensus.cc:348] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.144770 1316 raft_consensus.cc:374] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:33.144804 1316 raft_consensus.cc:729] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Initialized, Role: FOLLOWER
I20260811 05:08:33.144924 1316 consensus_queue.cc:260] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.145174 1316 ts_tablet_manager.cc:1428] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:33.145742 1317 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Bootstrap starting.
I20260811 05:08:33.146431 1375 ts_tablet_manager.cc:1918] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:33.146512 1375 log.cc:1196] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/6273699876744687a1697f1cc1b14935
I20260811 05:08:33.146714 1317 tablet_bootstrap.cc:654] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:33.146816 1375 ts_tablet_manager.cc:1939] T 6273699876744687a1697f1cc1b14935 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting consensus metadata
I20260811 05:08:33.147398 1314 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: No bootstrap required, opened a new log
I20260811 05:08:33.147485 1314 ts_tablet_manager.cc:1397] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 05:08:33.147873 1314 raft_consensus.cc:348] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.147974 1314 raft_consensus.cc:374] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:33.147979 1317 tablet_bootstrap.cc:492] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: No bootstrap required, opened a new log
I20260811 05:08:33.148003 1314 raft_consensus.cc:729] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Initialized, Role: FOLLOWER
I20260811 05:08:33.148033 1317 ts_tablet_manager.cc:1397] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:08:33.148118 1314 consensus_queue.cc:260] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.148371 1314 ts_tablet_manager.cc:1428] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:08:33.148461 1317 raft_consensus.cc:348] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.148584 1317 raft_consensus.cc:374] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:33.148617 1317 raft_consensus.cc:729] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Initialized, Role: FOLLOWER
I20260811 05:08:33.148734 1317 consensus_queue.cc:260] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.149012 1317 ts_tablet_manager.cc:1428] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:33.150003 673 catalog_manager.cc:4841] TS 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605): tablet 6273699876744687a1697f1cc1b14935 (table test-table [id=2dcf9fa9dba1442cb327fb79f050d138]) successfully deleted
I20260811 05:08:33.150264 675 catalog_manager.cc:4841] TS 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609): tablet 6273699876744687a1697f1cc1b14935 (table test-table [id=2dcf9fa9dba1442cb327fb79f050d138]) successfully deleted
I20260811 05:08:33.168183 1325 raft_consensus.cc:482] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:33.168301 1325 raft_consensus.cc:504] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.169306 1325 leader_election.cc:288] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:33.174118 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.169632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:37799 (local address 127.30.88.195:44083)
0811 05:08:33.169757 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.169761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.170105 (+ 344us) server_negotiation.cc:407] Connection header received
0811 05:08:33.170286 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.170292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.170347 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.170424 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:33.170776 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.171573 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.172888 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.173283 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.173427 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.173490 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.173863 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.173974 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:08:33.174137 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.169449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34605 (local address 127.30.88.194:45669)
0811 05:08:33.169927 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.169942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.170044 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.170356 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.170359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.170371 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:33.170491 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.170496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.171401 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.171407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.172669 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.172680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.173713 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.173717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.173827 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.173849 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.173916 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.173987 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":115,"threads_started":1}
I20260811 05:08:33.174584 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.169766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:45669 (local address 127.30.88.193:34605)
0811 05:08:33.170025 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.170029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.170043 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:33.170189 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.170192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.170260 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.170345 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:33.170598 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.171291 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.172833 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.173484 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.173608 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.173655 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.174322 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.174433 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260811 05:08:33.174618 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.169570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.30.88.194:37799)
0811 05:08:33.170008 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.170022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.170171 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.170494 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.170500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.170518 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:33.170654 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.170660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.171699 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.171702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.172774 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.172784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.173593 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.173597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.173704 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.173735 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.174378 (+ 643us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.174473 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":100,"threads_started":1}
I20260811 05:08:33.174988 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9526b63c12ae483b894217708d5bd59e" candidate_uuid: "1aff25f1226e4f80b97b748978954704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c" is_pre_election: true
I20260811 05:08:33.175155 881 raft_consensus.cc:2454] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aff25f1226e4f80b97b748978954704 in term 0.
I20260811 05:08:33.175336 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9526b63c12ae483b894217708d5bd59e" candidate_uuid: "1aff25f1226e4f80b97b748978954704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" is_pre_election: true
I20260811 05:08:33.175570 1224 raft_consensus.cc:2454] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aff25f1226e4f80b97b748978954704 in term 0.
I20260811 05:08:33.176012 970 leader_election.cc:302] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, 6c62631d7fee4d8caef70ef019f83b0c, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:33.176296 1325 raft_consensus.cc:2781] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:33.176366 1325 raft_consensus.cc:482] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:33.176432 1325 raft_consensus.cc:3037] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.177464 1325 raft_consensus.cc:504] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.177898 1325 leader_election.cc:288] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [CANDIDATE]: Term 1 election: Requested vote from peers 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:33.178194 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9526b63c12ae483b894217708d5bd59e" candidate_uuid: "1aff25f1226e4f80b97b748978954704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
I20260811 05:08:33.178189 881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9526b63c12ae483b894217708d5bd59e" candidate_uuid: "1aff25f1226e4f80b97b748978954704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c62631d7fee4d8caef70ef019f83b0c"
I20260811 05:08:33.178304 1224 raft_consensus.cc:3037] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.178333 881 raft_consensus.cc:3037] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:33.179292 881 raft_consensus.cc:2454] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aff25f1226e4f80b97b748978954704 in term 1.
I20260811 05:08:33.179459 1224 raft_consensus.cc:2454] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aff25f1226e4f80b97b748978954704 in term 1.
I20260811 05:08:33.179570 966 leader_election.cc:302] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, 6c62631d7fee4d8caef70ef019f83b0c; no voters:
I20260811 05:08:33.179760 1325 raft_consensus.cc:2781] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:33.180001 1325 raft_consensus.cc:686] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 1 LEADER]: Becoming Leader. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Running, Role: LEADER
I20260811 05:08:33.180241 1325 consensus_queue.cc:237] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:33.182390 686 catalog_manager.cc:5495] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aff25f1226e4f80b97b748978954704 (127.30.88.194). New cstate: current_term: 1 leader_uuid: "1aff25f1226e4f80b97b748978954704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:33.202942 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.197973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.30.88.194:42609)
0811 05:08:33.198256 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.198264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.198276 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:08:33.198373 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.198376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.198424 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.198554 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.198836 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.199764 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.201093 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.201560 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.201690 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.201988 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.202452 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.202589 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.202656 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.202710 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260811 05:08:33.218451 1354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.214696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:34201 (local address 127.30.88.195:44083)
0811 05:08:33.214809 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.214813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.214827 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:33.214890 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.214893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.214938 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.215014 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.215289 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.216008 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.217321 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.217645 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.217759 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.218061 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.218132 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.218198 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.218293 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.218339 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:08:33.218538 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.214346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.30.88.194:34201)
0811 05:08:33.214678 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.214694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.214800 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.215040 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.215043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.215052 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:33.215173 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.215178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.216131 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.216137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.217187 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.217198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.217770 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.217773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.217867 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.217916 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.218217 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.218223 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.218284 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.218355 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":120,"threads_started":1}
I20260811 05:08:33.224449 881 raft_consensus.cc:1261] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Refusing update from remote peer 1aff25f1226e4f80b97b748978954704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:33.224891 1383 consensus_queue.cc:1035] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.225143 1224 raft_consensus.cc:1261] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Refusing update from remote peer 1aff25f1226e4f80b97b748978954704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:33.225572 1325 consensus_queue.cc:1035] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:33.229696 1356 mvcc.cc:205] Tried to move back new op lower bound from 7317196444562362368 to 7317196444388737024. Current Snapshot: MvccSnapshot[applied={T|T < 7317196444562362368}]
I20260811 05:08:33.234452 1352 mvcc.cc:205] Tried to move back new op lower bound from 7317196444562362368 to 7317196444388737024. Current Snapshot: MvccSnapshot[applied={T|T < 7317196444562362368}]
I20260811 05:08:33.247628 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.243787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:42609 (local address 127.30.88.195:33509)
0811 05:08:33.243932 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:33.243945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:33.244016 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:33.244250 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:33.244254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:33.244263 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:33.244361 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.244367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.245339 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.245342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.246238 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:33.246248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.246842 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.246866 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:33.246961 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.246993 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.247319 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.247327 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:08:33.247409 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:08:33.247494 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 05:08:33.247622 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:33.243859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:33509 (local address 127.30.88.194:42609)
0811 05:08:33.243986 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:33.243991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:33.244019 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:08:33.244102 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:33.244104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:33.244142 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:33.244236 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:33.244478 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.245226 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.246357 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:33.246722 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:33.246836 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:33.247128 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:33.247215 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:33.247299 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:33.247424 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 05:08:33.247473 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 05:08:33.248335 1351 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:08:34.805760 687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:34.805943 687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:34.808310 687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Sending DeleteTablet for 3 replicas of tablet 9526b63c12ae483b894217708d5bd59e
I20260811 05:08:34.808920 1202 tablet_service.cc:1514] Processing DeleteTablet for tablet 9526b63c12ae483b894217708d5bd59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:34 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:59660
I20260811 05:08:34.808913 858 tablet_service.cc:1514] Processing DeleteTablet for tablet 9526b63c12ae483b894217708d5bd59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:34 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60670
I20260811 05:08:34.808938 1027 tablet_service.cc:1514] Processing DeleteTablet for tablet 9526b63c12ae483b894217708d5bd59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:34 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:56840
I20260811 05:08:34.809356 1421 tablet_replica.cc:330] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: stopping tablet replica
I20260811 05:08:34.809497 1421 raft_consensus.cc:2229] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:34.809616 1421 raft_consensus.cc:2258] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:34.809235 1422 tablet_replica.cc:330] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: stopping tablet replica
I20260811 05:08:34.809849 1420 tablet_replica.cc:330] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: stopping tablet replica
I20260811 05:08:34.809963 1420 raft_consensus.cc:2229] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:34.809967 1422 raft_consensus.cc:2229] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:34.810070 1420 raft_consensus.cc:2258] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:34.810303 1422 raft_consensus.cc:2258] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:34.810640 1421 ts_tablet_manager.cc:1905] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:34.810895 1422 ts_tablet_manager.cc:1905] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:34.813493 1422 ts_tablet_manager.cc:1918] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:08:34.813493 1421 ts_tablet_manager.cc:1918] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:08:34.813577 1422 log.cc:1196] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/9526b63c12ae483b894217708d5bd59e
I20260811 05:08:34.813577 1421 log.cc:1196] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/9526b63c12ae483b894217708d5bd59e
I20260811 05:08:34.813862 1422 ts_tablet_manager.cc:1939] T 9526b63c12ae483b894217708d5bd59e P 1aff25f1226e4f80b97b748978954704: Deleting consensus metadata
I20260811 05:08:34.813866 1421 ts_tablet_manager.cc:1939] T 9526b63c12ae483b894217708d5bd59e P bfc0d7fae98d4449adf317bfbed3c37d: Deleting consensus metadata
I20260811 05:08:34.814529 675 catalog_manager.cc:4841] TS 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609): tablet 9526b63c12ae483b894217708d5bd59e (table test-table [id=fb9b8683d92c4499b880bda29ad8604c]) successfully deleted
I20260811 05:08:34.814632 676 catalog_manager.cc:4841] TS bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083): tablet 9526b63c12ae483b894217708d5bd59e (table test-table [id=fb9b8683d92c4499b880bda29ad8604c]) successfully deleted
I20260811 05:08:34.814999 1420 ts_tablet_manager.cc:1905] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:34.817929 1420 ts_tablet_manager.cc:1918] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:08:34.817994 1420 log.cc:1196] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/9526b63c12ae483b894217708d5bd59e
I20260811 05:08:34.818233 1420 ts_tablet_manager.cc:1939] T 9526b63c12ae483b894217708d5bd59e P 6c62631d7fee4d8caef70ef019f83b0c: Deleting consensus metadata
I20260811 05:08:34.818799 673 catalog_manager.cc:4841] TS 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605): tablet 9526b63c12ae483b894217708d5bd59e (table test-table [id=fb9b8683d92c4499b880bda29ad8604c]) successfully deleted
Aug 11 05:08:34 dist-test-slave-7c35 krb5kdc[638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:08:34 dist-test-slave-7c35 krb5kdc[638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:34.837798 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.829748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.30.88.254:46099)
0811 05:08:34.829992 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:34.830000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:34.830057 (+ 57us) server_negotiation.cc:407] Connection header received
0811 05:08:34.830181 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:34.830184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:34.830225 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:34.830306 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:34.831014 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.831546 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.832327 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.832554 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.832650 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.834989 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:34.834991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:34.834993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:34.835016 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:34.836508 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:34.836850 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:34.836852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:34.836854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:34.836929 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:34.837169 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:34.837171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:34.837172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:34.837330 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:34.837441 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:34.837588 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 05:08:34.837677 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":104,"threads_started":1}
I20260811 05:08:34.840184 687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:34.840597 687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:34.845326 858 tablet_service.cc:1467] Processing CreateTablet for tablet 42e20ed5d253469b94998fb583ce4353 (DEFAULT_TABLE table=test-table [id=9ce6e3f2489942499c89d812089977d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:34.845399 1027 tablet_service.cc:1467] Processing CreateTablet for tablet 42e20ed5d253469b94998fb583ce4353 (DEFAULT_TABLE table=test-table [id=9ce6e3f2489942499c89d812089977d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:34.845675 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e20ed5d253469b94998fb583ce4353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:34.845675 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e20ed5d253469b94998fb583ce4353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:34.845634 1202 tablet_service.cc:1467] Processing CreateTablet for tablet 42e20ed5d253469b94998fb583ce4353 (DEFAULT_TABLE table=test-table [id=9ce6e3f2489942499c89d812089977d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:34.845952 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e20ed5d253469b94998fb583ce4353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:34.848197 1432 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Bootstrap starting.
I20260811 05:08:34.848214 1433 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Bootstrap starting.
I20260811 05:08:34.848356 1434 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Bootstrap starting.
I20260811 05:08:34.849080 1432 tablet_bootstrap.cc:654] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:34.849243 1434 tablet_bootstrap.cc:654] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:34.849481 1433 tablet_bootstrap.cc:654] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:34.850415 1432 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: No bootstrap required, opened a new log
I20260811 05:08:34.850472 1434 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: No bootstrap required, opened a new log
I20260811 05:08:34.850517 1432 ts_tablet_manager.cc:1397] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:34.850593 1434 ts_tablet_manager.cc:1397] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:34.850775 1433 tablet_bootstrap.cc:492] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: No bootstrap required, opened a new log
I20260811 05:08:34.850905 1433 ts_tablet_manager.cc:1397] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:34.850975 1432 raft_consensus.cc:348] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.851106 1432 raft_consensus.cc:374] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:34.851145 1432 raft_consensus.cc:729] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Initialized, Role: FOLLOWER
I20260811 05:08:34.851249 1434 raft_consensus.cc:348] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.851271 1432 consensus_queue.cc:260] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.851353 1434 raft_consensus.cc:374] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:34.851383 1434 raft_consensus.cc:729] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Initialized, Role: FOLLOWER
I20260811 05:08:34.851397 1433 raft_consensus.cc:348] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.851558 1432 ts_tablet_manager.cc:1428] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:34.851517 1434 consensus_queue.cc:260] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.851567 1433 raft_consensus.cc:374] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:34.851876 1434 ts_tablet_manager.cc:1428] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:34.851610 1433 raft_consensus.cc:729] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Initialized, Role: FOLLOWER
I20260811 05:08:34.852118 1433 consensus_queue.cc:260] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.852538 1433 ts_tablet_manager.cc:1428] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:34.949630 1440 raft_consensus.cc:482] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:34.949771 1440 raft_consensus.cc:504] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.950922 1440 leader_election.cc:288] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:34.955061 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.951261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:45205 (local address 127.30.88.194:42609)
0811 05:08:34.951634 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:34.951644 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:34.951770 (+ 126us) server_negotiation.cc:407] Connection header received
0811 05:08:34.951928 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:34.951931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:34.951981 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:34.952072 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:34.952331 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.952875 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954008 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.954410 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954548 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.954604 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:34.954820 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 05:08:34.954925 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":148,"threads_started":1}
I20260811 05:08:34.955061 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.951259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:40493 (local address 127.30.88.195:44083)
0811 05:08:34.951643 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:34.951652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:34.951667 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:08:34.951822 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:34.951824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:34.951866 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:34.951959 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:34.952247 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.952808 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954004 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.954411 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954548 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.954603 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:34.954862 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 05:08:34.954936 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":129,"threads_started":1}
I20260811 05:08:34.955070 1443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.951110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:42609 (local address 127.30.88.193:45205)
0811 05:08:34.951668 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:34.951690 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:34.951818 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:34.952083 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:34.952086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:34.952102 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:34.952220 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.952226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.952991 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.952995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.953874 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.953883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.954555 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.954704 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.954721 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:08:34.954804 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:08:34.954886 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":55,"thread_start_us":115,"threads_started":1}
I20260811 05:08:34.955082 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.951110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.30.88.193:40493)
0811 05:08:34.951545 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:34.951561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:34.951682 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:34.951970 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:34.951973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:34.951990 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:34.952128 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.952133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.952930 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.952934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.953862 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.953872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.954560 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.954564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.954723 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.954740 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:08:34.954824 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:08:34.954918 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":121,"thread_start_us":131,"threads_started":1}
I20260811 05:08:34.955596 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e20ed5d253469b94998fb583ce4353" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" is_pre_election: true
I20260811 05:08:34.955652 1049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e20ed5d253469b94998fb583ce4353" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704" is_pre_election: true
I20260811 05:08:34.955770 1224 raft_consensus.cc:2454] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 0.
I20260811 05:08:34.955801 1049 raft_consensus.cc:2454] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 0.
I20260811 05:08:34.956147 802 leader_election.cc:302] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, 6c62631d7fee4d8caef70ef019f83b0c; no voters:
I20260811 05:08:34.956434 1440 raft_consensus.cc:2781] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:34.956501 1440 raft_consensus.cc:482] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:34.956538 1440 raft_consensus.cc:3037] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:34.957504 1440 raft_consensus.cc:504] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.957837 1440 leader_election.cc:288] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 election: Requested vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:34.958181 1049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e20ed5d253469b94998fb583ce4353" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704"
I20260811 05:08:34.958200 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e20ed5d253469b94998fb583ce4353" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
I20260811 05:08:34.958293 1049 raft_consensus.cc:3037] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:34.958326 1224 raft_consensus.cc:3037] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:34.959369 1049 raft_consensus.cc:2454] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 1.
I20260811 05:08:34.959677 1224 raft_consensus.cc:2454] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 1.
I20260811 05:08:34.959733 802 leader_election.cc:302] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aff25f1226e4f80b97b748978954704, 6c62631d7fee4d8caef70ef019f83b0c; no voters:
I20260811 05:08:34.959925 1440 raft_consensus.cc:2781] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:34.960175 1440 raft_consensus.cc:686] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 LEADER]: Becoming Leader. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Running, Role: LEADER
I20260811 05:08:34.960444 1440 consensus_queue.cc:237] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:34.962544 690 catalog_manager.cc:5495] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193). New cstate: current_term: 1 leader_uuid: "6c62631d7fee4d8caef70ef019f83b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:34.977941 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.974446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.30.88.193:34605)
0811 05:08:34.974671 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:34.974677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:34.974688 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:08:34.974748 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:34.974750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:34.974793 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:34.974983 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:34.975197 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.975695 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.976462 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.976705 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.976778 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.977122 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:34.977624 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:34.977708 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:34.977785 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 05:08:34.977826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260811 05:08:34.987362 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.984520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38717 (local address 127.30.88.195:44083)
0811 05:08:34.984629 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:34.984632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:34.984745 (+ 113us) server_negotiation.cc:407] Connection header received
0811 05:08:34.984835 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:34.984837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:34.984873 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:34.984946 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:34.985170 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.985620 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.986410 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.986593 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.986657 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.986995 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:34.987053 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:34.987106 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:34.987231 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 05:08:34.987269 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:08:34.987442 1456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:34.984373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:44083 (local address 127.30.88.193:38717)
0811 05:08:34.984668 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:34.984681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:34.984768 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:34.984977 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:34.984979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:34.984985 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:34.985084 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.985089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.985709 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.985711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.986287 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:34.986295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:34.986685 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:34.986688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:34.986754 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:34.986793 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:34.987147 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:34.987152 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:08:34.987224 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:08:34.987287 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260811 05:08:34.993113 1049 raft_consensus.cc:1261] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Refusing update from remote peer 6c62631d7fee4d8caef70ef019f83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:34.993204 1224 raft_consensus.cc:1261] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Refusing update from remote peer 6c62631d7fee4d8caef70ef019f83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:34.993530 1440 consensus_queue.cc:1035] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:34.993729 1450 consensus_queue.cc:1035] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:35.011014 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:35.006441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:51245 (local address 127.30.88.193:34605)
0811 05:08:35.006553 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:35.006557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:35.007796 (+ 1239us) server_negotiation.cc:407] Connection header received
0811 05:08:35.007866 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:35.007869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:35.007912 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:35.007987 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:35.008211 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:35.008945 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:35.010096 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:35.010300 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:35.010379 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:35.010622 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:35.010678 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:35.010730 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:35.010832 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 05:08:35.010917 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:08:35.011049 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:35.006153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34605 (local address 127.30.88.195:51245)
0811 05:08:35.007660 (+ 1507us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:35.007672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:35.007802 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:35.007989 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:35.007991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:35.007995 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:08:35.008110 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:35.008115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:35.009071 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:35.009075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:35.009979 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:35.009987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:35.010408 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:35.010411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:35.010490 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:35.010518 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:08:35.010758 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:08:35.010763 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:08:35.010826 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:08:35.010917 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1434,"thread_start_us":89,"threads_started":1}
I20260811 05:08:35.031014 687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:35.031162 687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:35.033191 687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Sending DeleteTablet for 3 replicas of tablet 42e20ed5d253469b94998fb583ce4353
I20260811 05:08:35.033674 858 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e20ed5d253469b94998fb583ce4353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:35 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60670
I20260811 05:08:35.033852 1420 tablet_replica.cc:330] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: stopping tablet replica
I20260811 05:08:35.033841 1027 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e20ed5d253469b94998fb583ce4353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:35 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:56840
I20260811 05:08:35.033852 1202 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e20ed5d253469b94998fb583ce4353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:35 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:59660
I20260811 05:08:35.033967 1420 raft_consensus.cc:2229] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:35.034021 1421 tablet_replica.cc:330] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: stopping tablet replica
I20260811 05:08:35.034036 1422 tablet_replica.cc:330] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: stopping tablet replica
I20260811 05:08:35.034116 1421 raft_consensus.cc:2229] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:35.034149 1422 raft_consensus.cc:2229] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:35.034209 1421 raft_consensus.cc:2258] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:35.034227 1420 raft_consensus.cc:2258] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:35.034250 1422 raft_consensus.cc:2258] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:35.034983 1421 ts_tablet_manager.cc:1905] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:35.034977 1422 ts_tablet_manager.cc:1905] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:35.035061 1420 ts_tablet_manager.cc:1905] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:35.037499 1421 ts_tablet_manager.cc:1918] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:35.037499 1422 ts_tablet_manager.cc:1918] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:35.037525 1420 ts_tablet_manager.cc:1918] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:08:35.037569 1422 log.cc:1196] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/42e20ed5d253469b94998fb583ce4353
I20260811 05:08:35.037569 1421 log.cc:1196] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/42e20ed5d253469b94998fb583ce4353
I20260811 05:08:35.037591 1420 log.cc:1196] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/42e20ed5d253469b94998fb583ce4353
I20260811 05:08:35.037833 1421 ts_tablet_manager.cc:1939] T 42e20ed5d253469b94998fb583ce4353 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting consensus metadata
I20260811 05:08:35.037833 1422 ts_tablet_manager.cc:1939] T 42e20ed5d253469b94998fb583ce4353 P 1aff25f1226e4f80b97b748978954704: Deleting consensus metadata
I20260811 05:08:35.037848 1420 ts_tablet_manager.cc:1939] T 42e20ed5d253469b94998fb583ce4353 P 6c62631d7fee4d8caef70ef019f83b0c: Deleting consensus metadata
I20260811 05:08:35.038409 676 catalog_manager.cc:4841] TS bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083): tablet 42e20ed5d253469b94998fb583ce4353 (table test-table [id=9ce6e3f2489942499c89d812089977d6]) successfully deleted
I20260811 05:08:35.038594 673 catalog_manager.cc:4841] TS 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605): tablet 42e20ed5d253469b94998fb583ce4353 (table test-table [id=9ce6e3f2489942499c89d812089977d6]) successfully deleted
I20260811 05:08:35.038975 675 catalog_manager.cc:4841] TS 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609): tablet 42e20ed5d253469b94998fb583ce4353 (table test-table [id=9ce6e3f2489942499c89d812089977d6]) successfully deleted
I20260811 05:08:35.039367 687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:35.039788 687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:35.044277 858 tablet_service.cc:1467] Processing CreateTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 (DEFAULT_TABLE table=test-table [id=638a34027d4040648ec44eba4c8abe04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:35.044534 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0fc0886035a4c3baed8cc566ed259d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:35.044499 1202 tablet_service.cc:1467] Processing CreateTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 (DEFAULT_TABLE table=test-table [id=638a34027d4040648ec44eba4c8abe04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:35.044816 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0fc0886035a4c3baed8cc566ed259d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:35.046497 1434 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Bootstrap starting.
I20260811 05:08:35.046648 1433 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: Bootstrap starting.
I20260811 05:08:35.047232 1027 tablet_service.cc:1467] Processing CreateTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 (DEFAULT_TABLE table=test-table [id=638a34027d4040648ec44eba4c8abe04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:35.047377 1434 tablet_bootstrap.cc:654] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:35.047492 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0fc0886035a4c3baed8cc566ed259d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:35.047536 1433 tablet_bootstrap.cc:654] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:35.048538 1434 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: No bootstrap required, opened a new log
I20260811 05:08:35.048610 1433 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: No bootstrap required, opened a new log
I20260811 05:08:35.048642 1434 ts_tablet_manager.cc:1397] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:35.048681 1433 ts_tablet_manager.cc:1397] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:35.049070 1434 raft_consensus.cc:348] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.049090 1433 raft_consensus.cc:348] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.049173 1434 raft_consensus.cc:374] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:35.049188 1433 raft_consensus.cc:374] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:35.049202 1434 raft_consensus.cc:729] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc0d7fae98d4449adf317bfbed3c37d, State: Initialized, Role: FOLLOWER
I20260811 05:08:35.049216 1433 raft_consensus.cc:729] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Initialized, Role: FOLLOWER
I20260811 05:08:35.049316 1434 consensus_queue.cc:260] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.049324 1433 consensus_queue.cc:260] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.049582 1434 ts_tablet_manager.cc:1428] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:35.049609 1433 ts_tablet_manager.cc:1428] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:35.050067 1432 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Bootstrap starting.
I20260811 05:08:35.052842 1432 tablet_bootstrap.cc:654] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:35.054085 1432 tablet_bootstrap.cc:492] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: No bootstrap required, opened a new log
I20260811 05:08:35.054193 1432 ts_tablet_manager.cc:1397] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 05:08:35.054525 1432 raft_consensus.cc:348] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.054633 1432 raft_consensus.cc:374] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:35.054675 1432 raft_consensus.cc:729] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aff25f1226e4f80b97b748978954704, State: Initialized, Role: FOLLOWER
I20260811 05:08:35.054803 1432 consensus_queue.cc:260] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.055167 1432 ts_tablet_manager.cc:1428] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:35.289464 1462 raft_consensus.cc:482] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:35.289579 1462 raft_consensus.cc:504] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.290004 1462 leader_election.cc:288] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:35.290529 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0fc0886035a4c3baed8cc566ed259d9" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" is_pre_election: true
I20260811 05:08:35.290529 1049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0fc0886035a4c3baed8cc566ed259d9" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704" is_pre_election: true
I20260811 05:08:35.290684 1224 raft_consensus.cc:2454] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 0.
I20260811 05:08:35.290725 1049 raft_consensus.cc:2454] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 0.
I20260811 05:08:35.291034 803 leader_election.cc:302] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c62631d7fee4d8caef70ef019f83b0c, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:35.291265 1462 raft_consensus.cc:2781] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:35.291376 1462 raft_consensus.cc:482] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:35.291436 1462 raft_consensus.cc:3037] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:35.292563 1462 raft_consensus.cc:504] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.292907 1462 leader_election.cc:288] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 election: Requested vote from peers 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609), bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083)
I20260811 05:08:35.293234 1224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0fc0886035a4c3baed8cc566ed259d9" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc0d7fae98d4449adf317bfbed3c37d"
I20260811 05:08:35.293354 1224 raft_consensus.cc:3037] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:35.294296 1224 raft_consensus.cc:2454] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 1.
I20260811 05:08:35.294591 803 leader_election.cc:302] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c62631d7fee4d8caef70ef019f83b0c, bfc0d7fae98d4449adf317bfbed3c37d; no voters:
I20260811 05:08:35.294571 1049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0fc0886035a4c3baed8cc566ed259d9" candidate_uuid: "6c62631d7fee4d8caef70ef019f83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aff25f1226e4f80b97b748978954704"
I20260811 05:08:35.294694 1049 raft_consensus.cc:3037] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:35.294782 1462 raft_consensus.cc:2781] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:35.294920 1462 raft_consensus.cc:686] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 LEADER]: Becoming Leader. State: Replica: 6c62631d7fee4d8caef70ef019f83b0c, State: Running, Role: LEADER
I20260811 05:08:35.295073 1462 consensus_queue.cc:237] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } }
I20260811 05:08:35.295831 1049 raft_consensus.cc:2454] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c62631d7fee4d8caef70ef019f83b0c in term 1.
I20260811 05:08:35.296859 687 catalog_manager.cc:5495] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193). New cstate: current_term: 1 leader_uuid: "6c62631d7fee4d8caef70ef019f83b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c62631d7fee4d8caef70ef019f83b0c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:35.355711 1224 raft_consensus.cc:1261] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Refusing update from remote peer 6c62631d7fee4d8caef70ef019f83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:35.355782 1049 raft_consensus.cc:1261] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Refusing update from remote peer 6c62631d7fee4d8caef70ef019f83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:35.356141 1440 consensus_queue.cc:1035] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aff25f1226e4f80b97b748978954704" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 42609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:35.356279 1462 consensus_queue.cc:1035] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc0d7fae98d4449adf317bfbed3c37d" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:08:35.367754 1400 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:08:36.915280 687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:36.915459 687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:36.917728 687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 446d3eaa846d45818e386cdad53454dc: Sending DeleteTablet for 3 replicas of tablet b0fc0886035a4c3baed8cc566ed259d9
I20260811 05:08:36.918277 858 tablet_service.cc:1514] Processing DeleteTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:36 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60670
I20260811 05:08:36.918428 1202 tablet_service.cc:1514] Processing DeleteTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:36 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:59660
I20260811 05:08:36.918603 1027 tablet_service.cc:1514] Processing DeleteTablet for tablet b0fc0886035a4c3baed8cc566ed259d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:36 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:56840
I20260811 05:08:36.918661 1501 tablet_replica.cc:330] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c: stopping tablet replica
I20260811 05:08:36.918762 1503 tablet_replica.cc:330] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: stopping tablet replica
I20260811 05:08:36.918802 1501 raft_consensus.cc:2229] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:36.918936 1504 tablet_replica.cc:330] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: stopping tablet replica
I20260811 05:08:36.918951 1503 raft_consensus.cc:2229] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:36.919054 1503 raft_consensus.cc:2258] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:36.919093 1504 raft_consensus.cc:2229] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:36.919103 1501 raft_consensus.cc:2258] T b0fc0886035a4c3baed8cc566ed259d9 P 6c62631d7fee4d8caef70ef019f83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:36.919242 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 780
I20260811 05:08:36.919247 1504 raft_consensus.cc:2258] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:36.923213 1504 ts_tablet_manager.cc:1905] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:36.924347 1503 ts_tablet_manager.cc:1905] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:36.926342 1503 ts_tablet_manager.cc:1918] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:08:36.926440 1503 log.cc:1196] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/b0fc0886035a4c3baed8cc566ed259d9
I20260811 05:08:36.926478 1504 ts_tablet_manager.cc:1918] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:08:36.926563 1504 log.cc:1196] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/b0fc0886035a4c3baed8cc566ed259d9
I20260811 05:08:36.926752 1503 ts_tablet_manager.cc:1939] T b0fc0886035a4c3baed8cc566ed259d9 P bfc0d7fae98d4449adf317bfbed3c37d: Deleting consensus metadata
I20260811 05:08:36.926885 1504 ts_tablet_manager.cc:1939] T b0fc0886035a4c3baed8cc566ed259d9 P 1aff25f1226e4f80b97b748978954704: Deleting consensus metadata
I20260811 05:08:36.927482 676 catalog_manager.cc:4841] TS bfc0d7fae98d4449adf317bfbed3c37d (127.30.88.195:44083): tablet b0fc0886035a4c3baed8cc566ed259d9 (table test-table [id=638a34027d4040648ec44eba4c8abe04]) successfully deleted
I20260811 05:08:36.927601 675 catalog_manager.cc:4841] TS 1aff25f1226e4f80b97b748978954704 (127.30.88.194:42609): tablet b0fc0886035a4c3baed8cc566ed259d9 (table test-table [id=638a34027d4040648ec44eba4c8abe04]) successfully deleted
W20260811 05:08:36.931751 673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:34605 (error 108)
I20260811 05:08:36.932009 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 946
I20260811 05:08:36.933667 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:36.933169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34605 (local address 127.0.0.1:60682)
0811 05:08:36.933443 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:36.933536 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:34605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
W20260811 05:08:36.933821 673 catalog_manager.cc:4584] TS 6c62631d7fee4d8caef70ef019f83b0c (127.30.88.193:34605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0fc0886035a4c3baed8cc566ed259d9: Network error: Client connection negotiation failed: client connection to 127.30.88.193:34605: connect: Connection refused (error 111)
I20260811 05:08:36.941043 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 1116
I20260811 05:08:36.949684 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 656
2026-08-11T05:08:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6875 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:37 dist-test-slave-7c35 krb5kdc[1510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:37 dist-test-slave-7c35 krb5kdc[1510](info): set up 2 sockets
Aug 11 05:08:37 dist-test-slave-7c35 krb5kdc[1510](info): commencing operation
krb5kdc: starting...
W20260811 05:08:38.032351 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:38.157203 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:42417
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:42417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:38.262527 1527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:38.262885 1527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:38.262984 1527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:38.266932 1527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:38.267036 1527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:38.267082 1527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:38.267122 1527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:38.267154 1527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:38.272017 1527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:42417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:42417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:38.272917 1527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:38.273892 1527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:38.282356 1535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:38.282397 1527 server_base.cc:1034] running on GCE node
W20260811 05:08:38.282364 1534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:38.282361 1537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:38.283247 1527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:38.284308 1527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:38.285483 1527 hybrid_clock.cc:648] HybridClock initialized: now 1786424918285460 us; error 36 us; skew 500 ppm
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:38.288056 1527 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:38.288877 1527 webserver.cc:460] Webserver started at http://127.30.88.254:44765/ using document root <none> and password file <none>
I20260811 05:08:38.289364 1527 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:38.289448 1527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:38.289656 1527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:38.291615 1527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "59358d1c65da472e9be671584403e000"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "a84447a04456a681f38ba3686d071068"
server_key_iv: "fb39374e3215b8cc825b8713be2268ce"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.292145 1527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "59358d1c65da472e9be671584403e000"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "a84447a04456a681f38ba3686d071068"
server_key_iv: "fb39374e3215b8cc825b8713be2268ce"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.295503 1527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:08:38.297643 1547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.298677 1527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:38.298825 1527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "59358d1c65da472e9be671584403e000"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "a84447a04456a681f38ba3686d071068"
server_key_iv: "fb39374e3215b8cc825b8713be2268ce"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.298967 1527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:38.319693 1527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:38.322269 1527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:38.322489 1527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:38.329357 1527 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:42417
I20260811 05:08:38.329386 1615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:42417 every 8 connection(s)
I20260811 05:08:38.330238 1527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:38.332856 1616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:38.333038 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 1527
I20260811 05:08:38.333158 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:38.333426 31075 external_mini_cluster.cc:1428] Setting key 826e6d8a6e7c8cabd9a18942472d3a42
I20260811 05:08:38.339918 1616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000: Bootstrap starting.
I20260811 05:08:38.342464 1616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000: Neither blocks nor log segments found. Creating new log.
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:38.343401 1616 log.cc:824] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:38.345373 1616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000: No bootstrap required, opened a new log
I20260811 05:08:38.347721 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:38.334765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49512 (local address 127.30.88.254:42417)
0811 05:08:38.335183 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:38.335200 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:38.335238 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:08:38.336236 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:38.336260 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:38.336650 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:38.337007 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:38.337681 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.339243 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.340004 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.340612 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.340761 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:38.343354 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:38.343360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:38.343375 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:38.343406 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:38.345099 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.345563 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.345575 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.345580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.345689 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.345976 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.345978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.345979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.346327 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:38.346476 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:38.346761 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 05:08:38.347035 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":106,"threads_started":1}
I20260811 05:08:38.349383 1616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } }
I20260811 05:08:38.349644 1616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:38.349730 1616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59358d1c65da472e9be671584403e000, State: Initialized, Role: FOLLOWER
I20260811 05:08:38.350214 1616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } }
I20260811 05:08:38.350338 1616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:38.350378 1616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:38.350448 1616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:38.351315 1616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } }
I20260811 05:08:38.351593 1616 leader_election.cc:302] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59358d1c65da472e9be671584403e000; no voters:
I20260811 05:08:38.351825 1616 leader_election.cc:288] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:38.351954 1623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:38.352196 1623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [term 1 LEADER]: Becoming Leader. State: Replica: 59358d1c65da472e9be671584403e000, State: Running, Role: LEADER
I20260811 05:08:38.352526 1623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } }
I20260811 05:08:38.352989 1616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:38.354220 1626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59358d1c65da472e9be671584403e000. Latest consensus state: current_term: 1 leader_uuid: "59358d1c65da472e9be671584403e000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } } }
I20260811 05:08:38.354355 1626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:38.354744 1636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:38.354635 1625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59358d1c65da472e9be671584403e000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59358d1c65da472e9be671584403e000" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42417 } } }
I20260811 05:08:38.355368 1625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:38.358047 1636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:38.363683 1636 catalog_manager.cc:1347] Generated new cluster ID: e74032b193dd411b9ae0520e60e59a8f
I20260811 05:08:38.363782 1636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:38.380306 1636 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:38.381193 1636 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:38.389313 1636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59358d1c65da472e9be671584403e000: Generated new TSK 0
I20260811 05:08:38.389894 1636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:38.425473 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:38.557711 1656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:38.557928 1656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:38.557964 1656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:38.561772 1656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:38.561820 1656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:38.561868 1656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:38.566583 1656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:38.567543 1656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:38.568424 1656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:38.578063 1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:38.578110 1665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:38.578150 1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:38.578274 1656 server_base.cc:1034] running on GCE node
I20260811 05:08:38.578989 1656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:38.579602 1656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:38.580755 1656 hybrid_clock.cc:648] HybridClock initialized: now 1786424918580727 us; error 41 us; skew 500 ppm
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:38.583405 1656 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:38.584225 1656 webserver.cc:460] Webserver started at http://127.30.88.193:44175/ using document root <none> and password file <none>
I20260811 05:08:38.584760 1656 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:38.584847 1656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:38.585069 1656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:38.587047 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "dd377fbe66ff4b2f93a7e5ea7fca951a"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "9534aa6a647fb1d319a73ee46377f447"
server_key_iv: "5554eafaf18aa9f163b6f24fbd3c2f69"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.587586 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "dd377fbe66ff4b2f93a7e5ea7fca951a"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "9534aa6a647fb1d319a73ee46377f447"
server_key_iv: "5554eafaf18aa9f163b6f24fbd3c2f69"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.591028 1656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:38.593297 1672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.594444 1656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:38.594574 1656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "dd377fbe66ff4b2f93a7e5ea7fca951a"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "9534aa6a647fb1d319a73ee46377f447"
server_key_iv: "5554eafaf18aa9f163b6f24fbd3c2f69"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.594677 1656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:38.625468 1656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:38.629772 1656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:38.629997 1656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:38.630762 1656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:38.631961 1656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:38.632067 1656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.632143 1656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:38.632177 1656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.644008 1656 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:45341
I20260811 05:08:38.644037 1816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:45341 every 8 connection(s)
I20260811 05:08:38.644881 1656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:38.653565 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 1656
I20260811 05:08:38.653685 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:38.654052 31075 external_mini_cluster.cc:1428] Setting key bf1e80404e559bf9338d14ce495dde6d
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:38.659307 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:38.646673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:43955 (local address 127.30.88.254:42417)
0811 05:08:38.646808 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:38.646814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:38.647655 (+ 841us) server_negotiation.cc:407] Connection header received
0811 05:08:38.649008 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:38.649011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:38.649080 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:38.649158 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:38.650345 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.650979 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.652123 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.652374 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.652485 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:38.655457 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:38.655461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:38.655463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:38.655491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:38.657132 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.657599 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.657605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.657610 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.657701 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.658097 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.658099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.658100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.658261 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:38.658333 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:38.658966 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 05:08:38.659111 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:08:38.660236 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:38.647073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:42417 (local address 127.30.88.193:43955)
0811 05:08:38.647486 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:38.647525 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:38.648792 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:38.649325 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:38.649331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:38.649677 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:38.650184 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:38.650198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.651110 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.651114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:38.651984 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:38.651994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.652535 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.652540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:38.652711 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:38.653337 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:38.653362 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:38.655266 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:38.657250 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:38.657254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:38.657265 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:38.657505 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:38.657809 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:38.657813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:38.657817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:38.657994 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:38.658357 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:38.658364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:38.658664 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 05:08:38.658906 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 05:08:38.659233 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":106,"threads_started":1}
I20260811 05:08:38.661353 1817 heartbeater.cc:347] Connected to a master server at 127.30.88.254:42417
I20260811 05:08:38.661608 1817 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:38.662043 1817 heartbeater.cc:511] Master 127.30.88.254:42417 requested a full tablet report, sending...
I20260811 05:08:38.663523 1567 ts_manager.cc:194] Registered new tserver with Master: dd377fbe66ff4b2f93a7e5ea7fca951a (127.30.88.193:45341)
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:38.664691 1567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:43955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:38.682161 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:38.790038 1823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:38.790298 1823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:38.790337 1823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:38.794083 1823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:38.794134 1823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:38.794200 1823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:38.799046 1823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:38.800017 1823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:38.800997 1823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:38.809957 1829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:38.809957 1830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:38.810143 1823 server_base.cc:1034] running on GCE node
W20260811 05:08:38.809973 1832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:38.810654 1823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:38.811254 1823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:38.812438 1823 hybrid_clock.cc:648] HybridClock initialized: now 1786424918812430 us; error 27 us; skew 500 ppm
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:38.815081 1823 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:38.815824 1823 webserver.cc:460] Webserver started at http://127.30.88.194:39081/ using document root <none> and password file <none>
I20260811 05:08:38.816367 1823 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:38.816455 1823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:38.816679 1823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:38.818580 1823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "520c22e0c6004622ac74b2455aca0508"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "e03d8631d7947c3ce4661c92997d2c04"
server_key_iv: "4ce266bc220915fdf918b579124a06d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.819152 1823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "520c22e0c6004622ac74b2455aca0508"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "e03d8631d7947c3ce4661c92997d2c04"
server_key_iv: "4ce266bc220915fdf918b579124a06d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.822564 1823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 05:08:38.824973 1840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.826134 1823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:38.826265 1823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "520c22e0c6004622ac74b2455aca0508"
format_stamp: "Formatted at 2026-08-11 05:08:38 on dist-test-slave-7c35"
server_key: "e03d8631d7947c3ce4661c92997d2c04"
server_key_iv: "4ce266bc220915fdf918b579124a06d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:38.826370 1823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:38.843350 1823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:38.846077 1823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:38.846275 1823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:38.846819 1823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:38.847702 1823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:38.847770 1823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.847839 1823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:38.847882 1823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:38.857149 1823 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:44469
I20260811 05:08:38.857169 1989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:44469 every 8 connection(s)
I20260811 05:08:38.858017 1823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:08:38 dist-test-slave-7c35 krb5kdc[1510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:38.868268 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 1823
I20260811 05:08:38.868376 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:38.868662 31075 external_mini_cluster.cc:1428] Setting key ca17ac1bfdbe5616ce4c36b8b357062e
I20260811 05:08:38.871146 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:38.859665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:57797 (local address 127.30.88.254:42417)
0811 05:08:38.859826 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:38.859831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:38.860631 (+ 800us) server_negotiation.cc:407] Connection header received
0811 05:08:38.861879 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:38.861881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:38.861925 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:38.861996 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:38.863102 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.863723 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.864774 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.864969 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.865038 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:38.867394 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:38.867398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:38.867404 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:38.867433 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:38.868928 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.869489 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.869492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.869494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.869592 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:38.869940 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:38.869945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:38.869947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:38.870206 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:38.870297 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:38.870924 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 05:08:38.871026 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:08:38.872269 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:38.860056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:42417 (local address 127.30.88.194:57797)
0811 05:08:38.860471 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:38.860512 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:38.861682 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:38.862143 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:38.862148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:38.862451 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:38.862934 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:38.862956 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.863846 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.863848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:38.864676 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:38.864684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:38.865081 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:38.865084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:38.865211 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:38.865798 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:38.865814 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:38.867211 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:38.869052 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:38.869057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:38.869067 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:38.869384 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:38.869683 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:38.869685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:38.869686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:38.869806 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:38.870329 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:38.870335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:38.870622 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 05:08:38.870833 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 05:08:38.871208 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260811 05:08:38.873399 1990 heartbeater.cc:347] Connected to a master server at 127.30.88.254:42417
I20260811 05:08:38.873641 1990 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:38.874095 1990 heartbeater.cc:511] Master 127.30.88.254:42417 requested a full tablet report, sending...
I20260811 05:08:38.875144 1567 ts_manager.cc:194] Registered new tserver with Master: 520c22e0c6004622ac74b2455aca0508 (127.30.88.194:44469)
I20260811 05:08:38.875960 1567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:57797
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:38.895823 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:39.005245 1995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:39.005484 1995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:39.005522 1995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:39.009687 1995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:39.009737 1995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:39.009826 1995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:39.014932 1995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:42417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:39.015910 1995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:39.016882 1995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:39.026540 2001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:39.026548 2006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:39.026598 2002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:39.026927 1995 server_base.cc:1034] running on GCE node
I20260811 05:08:39.027418 1995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:39.028015 1995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:39.029211 1995 hybrid_clock.cc:648] HybridClock initialized: now 1786424919029198 us; error 25 us; skew 500 ppm
Aug 11 05:08:39 dist-test-slave-7c35 krb5kdc[1510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424919, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:39.032015 1995 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:39.032778 1995 webserver.cc:460] Webserver started at http://127.30.88.195:41877/ using document root <none> and password file <none>
I20260811 05:08:39.033326 1995 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:39.033417 1995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:39.033644 1995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:39.035625 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "abf4ee995e264d619db2232c916aed2a"
format_stamp: "Formatted at 2026-08-11 05:08:39 on dist-test-slave-7c35"
server_key: "1e4ce994c4aa2329fc408ac1b022d641"
server_key_iv: "b534c58a4f432e105f8a08559146cdbb"
server_key_version: "encryptionkey@0"
I20260811 05:08:39.036175 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "abf4ee995e264d619db2232c916aed2a"
format_stamp: "Formatted at 2026-08-11 05:08:39 on dist-test-slave-7c35"
server_key: "1e4ce994c4aa2329fc408ac1b022d641"
server_key_iv: "b534c58a4f432e105f8a08559146cdbb"
server_key_version: "encryptionkey@0"
I20260811 05:08:39.039711 1995 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:39.041994 2016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:39.043244 1995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:39.043380 1995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "abf4ee995e264d619db2232c916aed2a"
format_stamp: "Formatted at 2026-08-11 05:08:39 on dist-test-slave-7c35"
server_key: "1e4ce994c4aa2329fc408ac1b022d641"
server_key_iv: "b534c58a4f432e105f8a08559146cdbb"
server_key_version: "encryptionkey@0"
I20260811 05:08:39.043488 1995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:39.055817 1995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:39.059046 1995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:39.059237 1995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:39.059762 1995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:39.060511 1995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:39.060577 1995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:39.060639 1995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:39.060691 1995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:39.069342 1995 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39355
I20260811 05:08:39.069366 2156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39355 every 8 connection(s)
I20260811 05:08:39.070196 1995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:39.071815 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 1995
I20260811 05:08:39.071938 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:39.072228 31075 external_mini_cluster.cc:1428] Setting key 3466c3beee800903d66aa0eb9a08fc6b
Aug 11 05:08:39 dist-test-slave-7c35 krb5kdc[1510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424919, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:39.083334 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:39.072007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:42057 (local address 127.30.88.254:42417)
0811 05:08:39.072166 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:39.072171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:39.072851 (+ 680us) server_negotiation.cc:407] Connection header received
0811 05:08:39.074301 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:39.074304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:39.074360 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:39.074452 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:39.075669 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:39.076180 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:39.077503 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:39.077884 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:39.078009 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:39.080282 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:39.080284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:39.080286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:39.080308 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:39.081526 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:39.081987 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:39.081990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:39.081991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:39.082065 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:39.082355 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:39.082357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:39.082358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:39.082535 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:39.082610 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:39.083120 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 05:08:39.083215 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:08:39.084138 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:39.072304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:42417 (local address 127.30.88.195:42057)
0811 05:08:39.072727 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:39.072773 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:39.074082 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:39.074615 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:39.074621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:39.074994 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:39.075525 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:39.075538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:39.076308 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:39.076316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:39.077350 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:39.077365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:39.078079 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:39.078082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:39.078214 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:39.078690 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:39.078705 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:39.080135 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:39.081637 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:39.081642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:39.081652 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:39.081897 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:39.082146 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:39.082148 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:39.082149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:39.082274 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:39.082622 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:39.082627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:39.082841 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 05:08:39.083085 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 05:08:39.083322 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":125,"threads_started":1}
I20260811 05:08:39.085238 2158 heartbeater.cc:347] Connected to a master server at 127.30.88.254:42417
I20260811 05:08:39.085487 2158 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:39.086045 2158 heartbeater.cc:511] Master 127.30.88.254:42417 requested a full tablet report, sending...
I20260811 05:08:39.087085 1565 ts_manager.cc:194] Registered new tserver with Master: abf4ee995e264d619db2232c916aed2a (127.30.88.195:39355)
I20260811 05:08:39.087652 1565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:42057
I20260811 05:08:39.097410 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:08:39.111521 1620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:39.110182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.30.88.254:42417)
0811 05:08:39.110307 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:39.110310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:39.110512 (+ 202us) server_negotiation.cc:407] Connection header received
0811 05:08:39.110616 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:39.110619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:39.110655 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:39.110725 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:39.111393 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49526: BlockingRecv error: recv got EOF from 127.0.0.1:49526 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:08:39.112756 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 1656
I20260811 05:08:39.121362 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 1823
I20260811 05:08:39.130723 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 1995
I20260811 05:08:39.138794 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 1527
2026-08-11T05:08:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2188 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:39 dist-test-slave-7c35 krb5kdc[2173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:39 dist-test-slave-7c35 krb5kdc[2173](info): set up 2 sockets
Aug 11 05:08:39 dist-test-slave-7c35 krb5kdc[2173](info): commencing operation
krb5kdc: starting...
W20260811 05:08:40.226236 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:40.357409 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38961
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:40.466658 2187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:40.466935 2187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:40.467041 2187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:40.470968 2187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:40.471040 2187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:40.471081 2187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:40.471117 2187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:40.471146 2187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:40.476276 2187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:40.477222 2187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:40.478180 2187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:40.486469 2192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:40.486438 2195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:40.486426 2193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:40.486832 2187 server_base.cc:1034] running on GCE node
I20260811 05:08:40.487412 2187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:40.488444 2187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:40.489629 2187 hybrid_clock.cc:648] HybridClock initialized: now 1786424920489603 us; error 42 us; skew 500 ppm
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:40.492377 2187 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:40.493155 2187 webserver.cc:460] Webserver started at http://127.30.88.254:39063/ using document root <none> and password file <none>
I20260811 05:08:40.493672 2187 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:40.493731 2187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:40.493886 2187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:40.495820 2187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "d3d903410912417b91ea7cff18b3e7a3"
format_stamp: "Formatted at 2026-08-11 05:08:40 on dist-test-slave-7c35"
server_key: "b59ad80c824a4de524456f214e5acabf"
server_key_iv: "1a058e30de311834b6e2b37b8079ece1"
server_key_version: "encryptionkey@0"
I20260811 05:08:40.496312 2187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "d3d903410912417b91ea7cff18b3e7a3"
format_stamp: "Formatted at 2026-08-11 05:08:40 on dist-test-slave-7c35"
server_key: "b59ad80c824a4de524456f214e5acabf"
server_key_iv: "1a058e30de311834b6e2b37b8079ece1"
server_key_version: "encryptionkey@0"
I20260811 05:08:40.499814 2187 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:08:40.501998 2203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:40.503198 2187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:40.503336 2187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "d3d903410912417b91ea7cff18b3e7a3"
format_stamp: "Formatted at 2026-08-11 05:08:40 on dist-test-slave-7c35"
server_key: "b59ad80c824a4de524456f214e5acabf"
server_key_iv: "1a058e30de311834b6e2b37b8079ece1"
server_key_version: "encryptionkey@0"
I20260811 05:08:40.503433 2187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:40.519315 2187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:40.521947 2187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:40.522143 2187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:40.528687 2187 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38961
I20260811 05:08:40.528955 2269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38961 every 8 connection(s)
I20260811 05:08:40.529572 2187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:40.532271 2271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:40.532938 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2187
I20260811 05:08:40.533068 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:40.533362 31075 external_mini_cluster.cc:1428] Setting key 9fb0f226a86067cf0e6f450b6470e095
I20260811 05:08:40.538687 2271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3: Bootstrap starting.
I20260811 05:08:40.540995 2271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:40.541695 2271 log.cc:824] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:40.543959 2271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3: No bootstrap required, opened a new log
I20260811 05:08:40.547457 2271 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } }
I20260811 05:08:40.547616 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:40.534684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.30.88.254:38961)
0811 05:08:40.535140 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:40.535158 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:40.535200 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:08:40.535964 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:40.535986 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:40.536367 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:40.536694 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:40.537334 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:40.538658 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:40.539541 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:40.540178 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:40.540338 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:40.543066 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:40.543080 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:40.543096 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:40.543132 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:40.544800 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:40.545265 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:40.545270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:40.545274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:40.545379 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:40.545674 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:40.545678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:40.545679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:40.546071 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:40.546271 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:40.546577 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 05:08:40.546881 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":169,"threads_started":1}
I20260811 05:08:40.547670 2271 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:40.547740 2271 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d903410912417b91ea7cff18b3e7a3, State: Initialized, Role: FOLLOWER
I20260811 05:08:40.548161 2271 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } }
I20260811 05:08:40.548343 2271 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:40.548432 2271 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:40.548580 2271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:40.549683 2271 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } }
I20260811 05:08:40.550040 2271 leader_election.cc:302] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3d903410912417b91ea7cff18b3e7a3; no voters:
I20260811 05:08:40.550431 2271 leader_election.cc:288] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:40.550498 2280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:40.550698 2280 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [term 1 LEADER]: Becoming Leader. State: Replica: d3d903410912417b91ea7cff18b3e7a3, State: Running, Role: LEADER
I20260811 05:08:40.550776 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:08:40.551061 2280 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } }
I20260811 05:08:40.551685 2271 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:40.552202 2280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3d903410912417b91ea7cff18b3e7a3. Latest consensus state: current_term: 1 leader_uuid: "d3d903410912417b91ea7cff18b3e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } } }
I20260811 05:08:40.552304 2280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:40.552639 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3d903410912417b91ea7cff18b3e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3d903410912417b91ea7cff18b3e7a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38961 } } }
I20260811 05:08:40.552760 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:40.552706 2287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:40.555858 2287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:40.561676 2287 catalog_manager.cc:1347] Generated new cluster ID: b06e7c0160be46a4a18710b66ad1358d
I20260811 05:08:40.561735 2287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:40.568878 2287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:40.569949 2287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:40.571342 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:40.565854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.30.88.254:38961)
0811 05:08:40.565970 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:40.565975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:40.566245 (+ 270us) server_negotiation.cc:407] Connection header received
0811 05:08:40.566405 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:40.566408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:40.566466 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:40.566562 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:40.567171 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:40.567690 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:40.568906 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:40.569265 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:40.569383 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:40.570302 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:40.570306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:40.570309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:40.570813 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:40.570844 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:40.570892 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:40.570975 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:40.571112 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 05:08:40.571186 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:08:40.578445 2287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3d903410912417b91ea7cff18b3e7a3: Generated new TSK 0
I20260811 05:08:40.579157 2287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:08:40.585623 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2187
2026-08-11T05:08:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1448 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2314](info): set up 2 sockets
Aug 11 05:08:40 dist-test-slave-7c35 krb5kdc[2314](info): commencing operation
krb5kdc: starting...
W20260811 05:08:41.675721 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:41 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:41.794183 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34767
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:34767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:41.899580 2329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:41.899842 2329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:41.899928 2329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:41.903730 2329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:41.903816 2329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:41.903858 2329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:41.903918 2329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:41.903959 2329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:41.908903 2329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:34767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:41.909879 2329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:41.910830 2329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:41.918766 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:41.918862 2329 server_base.cc:1034] running on GCE node
W20260811 05:08:41.918906 2335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:41.918777 2339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:41.919531 2329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:41.920564 2329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:41.921722 2329 hybrid_clock.cc:648] HybridClock initialized: now 1786424921921705 us; error 30 us; skew 500 ppm
Aug 11 05:08:41 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424921, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:41.924348 2329 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:41.925151 2329 webserver.cc:460] Webserver started at http://127.30.88.254:40573/ using document root <none> and password file <none>
I20260811 05:08:41.925647 2329 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:41.925701 2329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:41.925853 2329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:41.927719 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "d89a7df99e5f4b77a15cb06f47004bd2"
format_stamp: "Formatted at 2026-08-11 05:08:41 on dist-test-slave-7c35"
server_key: "28e8734b8f9b84e9dc1ac622646d17e7"
server_key_iv: "d75bc8222e8012bd1e6cc8b75ea970e4"
server_key_version: "encryptionkey@0"
I20260811 05:08:41.928236 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "d89a7df99e5f4b77a15cb06f47004bd2"
format_stamp: "Formatted at 2026-08-11 05:08:41 on dist-test-slave-7c35"
server_key: "28e8734b8f9b84e9dc1ac622646d17e7"
server_key_iv: "d75bc8222e8012bd1e6cc8b75ea970e4"
server_key_version: "encryptionkey@0"
I20260811 05:08:41.931679 2329 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:41.933832 2348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:41.935050 2329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:41.935158 2329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "d89a7df99e5f4b77a15cb06f47004bd2"
format_stamp: "Formatted at 2026-08-11 05:08:41 on dist-test-slave-7c35"
server_key: "28e8734b8f9b84e9dc1ac622646d17e7"
server_key_iv: "d75bc8222e8012bd1e6cc8b75ea970e4"
server_key_version: "encryptionkey@0"
I20260811 05:08:41.935243 2329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:41.979323 2329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:41.982062 2329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:41.982259 2329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:41.988993 2329 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:34767
I20260811 05:08:41.989274 2414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:34767 every 8 connection(s)
I20260811 05:08:41.989935 2329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:41.992542 2417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:41.997999 2417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2: Bootstrap starting.
I20260811 05:08:42.000068 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2329
I20260811 05:08:42.000216 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:42.000294 2417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:42.000521 31075 external_mini_cluster.cc:1428] Setting key 02c25961a5b1aec3f630ec084e473dcd
I20260811 05:08:42.001143 2417 log.cc:824] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:42.002982 2417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2: No bootstrap required, opened a new log
I20260811 05:08:42.006189 2417 raft_consensus.cc:348] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } }
I20260811 05:08:42.006387 2417 raft_consensus.cc:374] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:42.006458 2417 raft_consensus.cc:729] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d89a7df99e5f4b77a15cb06f47004bd2, State: Initialized, Role: FOLLOWER
I20260811 05:08:42.006958 2417 consensus_queue.cc:260] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } }
I20260811 05:08:42.007081 2417 raft_consensus.cc:388] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:42.007161 2417 raft_consensus.cc:482] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:42.007274 2417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:42.008082 2417 raft_consensus.cc:504] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } }
I20260811 05:08:42.008396 2417 leader_election.cc:302] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d89a7df99e5f4b77a15cb06f47004bd2; no voters:
I20260811 05:08:42.008669 2417 leader_election.cc:288] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:42.008787 2424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:42.009021 2424 raft_consensus.cc:686] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [term 1 LEADER]: Becoming Leader. State: Replica: d89a7df99e5f4b77a15cb06f47004bd2, State: Running, Role: LEADER
I20260811 05:08:42.009383 2424 consensus_queue.cc:237] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } }
I20260811 05:08:42.009829 2417 sys_catalog.cc:564] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:42.010999 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } } }
I20260811 05:08:42.011139 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:42.011535 2434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:42.011490 2426 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d89a7df99e5f4b77a15cb06f47004bd2. Latest consensus state: current_term: 1 leader_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89a7df99e5f4b77a15cb06f47004bd2" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34767 } } }
I20260811 05:08:42.011881 2426 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2 [sys.catalog]: This master's current role is: LEADER
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:42.014783 2434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:42.019733 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.001571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53954 (local address 127.30.88.254:34767)
0811 05:08:42.001991 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:42.002005 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:42.002037 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:08:42.002638 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:42.002653 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:42.003015 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:42.003384 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:42.004197 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.005159 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.005994 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.006525 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.006734 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.014157 (+ 7423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:42.014164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:42.014179 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:42.014209 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:42.016373 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.016863 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.016871 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.016876 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.017005 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.017302 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.017304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.017305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.017854 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:42.018078 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:42.018574 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 05:08:42.018879 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260811 05:08:42.020638 2434 catalog_manager.cc:1347] Generated new cluster ID: 6dac21f787ca4afc875b8e99dc1d6acb
I20260811 05:08:42.020694 2434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:42.058539 2434 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:42.059526 2434 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:42.071671 2434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d89a7df99e5f4b77a15cb06f47004bd2: Generated new TSK 0
I20260811 05:08:42.072346 2434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:42.105413 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:42.233763 2449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:42.233995 2449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:42.234036 2449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:42.237969 2449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:42.238019 2449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:42.238095 2449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:42.243144 2449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:42.244088 2449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:42.245039 2449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:42.254251 2458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:42.254267 2449 server_base.cc:1034] running on GCE node
W20260811 05:08:42.254254 2455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:42.254256 2454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:42.254980 2449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:42.255488 2449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:42.256685 2449 hybrid_clock.cc:648] HybridClock initialized: now 1786424922256676 us; error 44 us; skew 500 ppm
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:42.259258 2449 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:42.260099 2449 webserver.cc:460] Webserver started at http://127.30.88.193:39791/ using document root <none> and password file <none>
I20260811 05:08:42.260622 2449 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:42.260677 2449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:42.260893 2449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:42.262715 2449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "c2c9c750eab141aaae97bdd8d47e5852"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "b99a9e07084800bc70cbd2e51f31f45f"
server_key_iv: "7662dbe66efbf14720437162d1d38483"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.263332 2449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "c2c9c750eab141aaae97bdd8d47e5852"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "b99a9e07084800bc70cbd2e51f31f45f"
server_key_iv: "7662dbe66efbf14720437162d1d38483"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.266721 2449 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:42.268898 2465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.270056 2449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:42.270191 2449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "c2c9c750eab141aaae97bdd8d47e5852"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "b99a9e07084800bc70cbd2e51f31f45f"
server_key_iv: "7662dbe66efbf14720437162d1d38483"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.270284 2449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:42.280707 2449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:42.283236 2449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:42.283460 2449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:42.284034 2449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:42.284801 2449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:42.284854 2449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.284929 2449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:42.284958 2449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.292538 2449 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:38453
I20260811 05:08:42.292572 2607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:38453 every 8 connection(s)
I20260811 05:08:42.293255 2449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:42.301939 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2449
I20260811 05:08:42.302057 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:42.302359 31075 external_mini_cluster.cc:1428] Setting key 93b0b42d22622a965ae1f8cf351bde75
I20260811 05:08:42.306234 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.294910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38849 (local address 127.30.88.254:34767)
0811 05:08:42.295042 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:42.295046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:42.295744 (+ 698us) server_negotiation.cc:407] Connection header received
0811 05:08:42.297063 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:42.297066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:42.297121 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:42.297225 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:42.298276 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.298795 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.299926 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.300168 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.300272 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.302470 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:42.302474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:42.302476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:42.302506 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:42.304019 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.304634 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.304637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.304638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.304707 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.305063 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.305065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.305066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.305223 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:42.305295 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:42.305881 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 05:08:42.306023 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:08:42.306711 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.295216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34767 (local address 127.30.88.193:38849)
0811 05:08:42.295613 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:42.295649 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:42.296858 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:42.297389 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:42.297395 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:42.297703 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:42.298136 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.298148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.298973 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.298977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.299779 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.299788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.300304 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.300308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.300459 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.300879 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:42.300894 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:42.302294 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:42.304134 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.304144 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.304160 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.304506 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.304789 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.304794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.304798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.304973 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.305317 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:42.305325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:42.305599 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 05:08:42.305809 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 05:08:42.306020 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":113,"threads_started":1}
I20260811 05:08:42.308048 2608 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34767
I20260811 05:08:42.308259 2608 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:42.308733 2608 heartbeater.cc:511] Master 127.30.88.254:34767 requested a full tablet report, sending...
I20260811 05:08:42.310176 2369 ts_manager.cc:194] Registered new tserver with Master: c2c9c750eab141aaae97bdd8d47e5852 (127.30.88.193:38453)
I20260811 05:08:42.311477 2369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:38849
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:42.329319 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:42.435876 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:42.436106 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:42.436180 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:42.439915 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:42.439985 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:42.440062 2614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:42.444991 2614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:42.446023 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:42.447052 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:42.456172 2622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:42.456197 2620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:42.456228 2624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:42.456605 2614 server_base.cc:1034] running on GCE node
I20260811 05:08:42.457054 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:42.457753 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:42.458920 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786424922458892 us; error 38 us; skew 500 ppm
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:42.461651 2614 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:42.462381 2614 webserver.cc:460] Webserver started at http://127.30.88.194:44019/ using document root <none> and password file <none>
I20260811 05:08:42.462893 2614 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:42.462966 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:42.463128 2614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:42.464967 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "8d5d77f399cb466f823f38a648d01992"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "abfb94ef181a9eddfecda776167c1b11"
server_key_iv: "9fe0b06e07595c1695dea281c79035d1"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.465451 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "8d5d77f399cb466f823f38a648d01992"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "abfb94ef181a9eddfecda776167c1b11"
server_key_iv: "9fe0b06e07595c1695dea281c79035d1"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.468725 2614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:42.470911 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.472056 2614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:42.472189 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "8d5d77f399cb466f823f38a648d01992"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "abfb94ef181a9eddfecda776167c1b11"
server_key_iv: "9fe0b06e07595c1695dea281c79035d1"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.472287 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:42.483412 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:42.486614 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:42.486804 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:42.487390 2614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:42.488169 2614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:42.488214 2614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.488282 2614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:42.488325 2614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.496726 2614 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:33385
I20260811 05:08:42.496757 2775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:33385 every 8 connection(s)
I20260811 05:08:42.497507 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:42.505095 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2614
I20260811 05:08:42.505224 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:42.505515 31075 external_mini_cluster.cc:1428] Setting key 81d1bec53230b4f7d4e78d5c3c56313b
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:42.510500 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.499173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:56651 (local address 127.30.88.254:34767)
0811 05:08:42.499335 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:42.499340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:42.500071 (+ 731us) server_negotiation.cc:407] Connection header received
0811 05:08:42.501300 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:42.501302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:42.501348 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:42.501420 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:42.502468 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.503038 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.504112 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.504314 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.504407 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.506914 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:42.506918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:42.506920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:42.506970 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:42.508251 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.508828 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.508830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.508831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.508898 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.509278 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.509283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.509287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.509524 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:42.509623 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:42.510155 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 05:08:42.510288 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:08:42.511309 2778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.499506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34767 (local address 127.30.88.194:56651)
0811 05:08:42.499920 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:42.499964 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:42.501121 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:42.501596 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:42.501602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:42.501915 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:42.502334 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.502346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.503139 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.503141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.504010 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.504019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.504448 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.504451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.504581 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.505079 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:42.505099 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:42.506737 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:42.508368 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.508377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.508392 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.508730 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.508980 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.508985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.508989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.509172 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.509638 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:42.509642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:42.509880 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 05:08:42.510026 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 05:08:42.510305 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
I20260811 05:08:42.512452 2776 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34767
I20260811 05:08:42.512712 2776 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:42.513154 2776 heartbeater.cc:511] Master 127.30.88.254:34767 requested a full tablet report, sending...
I20260811 05:08:42.514230 2369 ts_manager.cc:194] Registered new tserver with Master: 8d5d77f399cb466f823f38a648d01992 (127.30.88.194:33385)
I20260811 05:08:42.514809 2369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:56651
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:42.534225 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:42.641794 2783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:42.642028 2783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:42.642066 2783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:42.645767 2783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:42.645830 2783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:42.645884 2783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:42.650645 2783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:42.651688 2783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:42.652695 2783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:42.661763 2794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:42.661782 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:42.661763 2788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:42.661950 2783 server_base.cc:1034] running on GCE node
I20260811 05:08:42.662433 2783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:42.663046 2783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:42.664201 2783 hybrid_clock.cc:648] HybridClock initialized: now 1786424922664182 us; error 27 us; skew 500 ppm
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:42.666972 2783 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:42.667832 2783 webserver.cc:460] Webserver started at http://127.30.88.195:40635/ using document root <none> and password file <none>
I20260811 05:08:42.668447 2783 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:42.668519 2783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:42.668753 2783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:42.670668 2783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "af035e637a1343c9a7aed47cf212a7cb"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "cc710110bb741a6165e80e760983f386"
server_key_iv: "6fb27d6ee72715d5fe7889eeb5e50cd8"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.671286 2783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "af035e637a1343c9a7aed47cf212a7cb"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "cc710110bb741a6165e80e760983f386"
server_key_iv: "6fb27d6ee72715d5fe7889eeb5e50cd8"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.674979 2783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:42.677266 2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.678380 2783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:42.678504 2783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "af035e637a1343c9a7aed47cf212a7cb"
format_stamp: "Formatted at 2026-08-11 05:08:42 on dist-test-slave-7c35"
server_key: "cc710110bb741a6165e80e760983f386"
server_key_iv: "6fb27d6ee72715d5fe7889eeb5e50cd8"
server_key_version: "encryptionkey@0"
I20260811 05:08:42.678603 2783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:42.702538 2783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:42.705739 2783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:42.705940 2783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:42.706496 2783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:42.707327 2783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:42.707394 2783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.707458 2783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:42.707500 2783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:42.716472 2783 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:33445
I20260811 05:08:42.716516 2940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:33445 every 8 connection(s)
I20260811 05:08:42.717286 2783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:42.720369 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2783
I20260811 05:08:42.720511 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:42.720860 31075 external_mini_cluster.cc:1428] Setting key e65b2b3a915e304b4fc2245c23a9d9ac
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:42.730027 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.718953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:49665 (local address 127.30.88.254:34767)
0811 05:08:42.719108 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:42.719113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:42.719893 (+ 780us) server_negotiation.cc:407] Connection header received
0811 05:08:42.721616 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:42.721619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:42.721681 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:42.721757 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:42.723203 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.723736 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.724815 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.725012 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.725099 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.727334 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:42.727337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:42.727339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:42.727359 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:42.728491 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.728905 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.728907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.728908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.728974 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.729239 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.729242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.729243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.729387 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:42.729464 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:42.729838 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 05:08:42.729923 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:08:42.730752 2944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.719331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34767 (local address 127.30.88.195:49665)
0811 05:08:42.719745 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:42.719783 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:42.721408 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:42.721934 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:42.721940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:42.722409 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:42.723054 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.723074 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.723840 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.723842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.724694 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:42.724703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.725141 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.725144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:42.725268 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.725809 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:42.725823 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:42.727195 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:42.728587 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.728591 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.728601 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.728819 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.729044 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:42.729046 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:42.729047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:42.729159 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:42.729464 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:42.729468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:42.729653 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 05:08:42.729780 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:08:42.729995 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":113,"threads_started":1}
I20260811 05:08:42.731801 2941 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34767
I20260811 05:08:42.732021 2941 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:42.732465 2941 heartbeater.cc:511] Master 127.30.88.254:34767 requested a full tablet report, sending...
I20260811 05:08:42.733382 2369 ts_manager.cc:194] Registered new tserver with Master: af035e637a1343c9a7aed47cf212a7cb (127.30.88.195:33445)
I20260811 05:08:42.734041 2369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:49665
I20260811 05:08:42.734772 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424922, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:42.759943 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:42.751978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53970 (local address 127.30.88.254:34767)
0811 05:08:42.752142 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:42.752146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:42.752249 (+ 103us) server_negotiation.cc:407] Connection header received
0811 05:08:42.752334 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:42.752336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:42.752371 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:42.752453 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:42.753080 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.753582 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.754348 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:42.754557 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:42.754631 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:42.756950 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:42.756953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:42.756955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:42.756983 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:42.758359 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.758735 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.758737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.758738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.758808 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:42.759091 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:42.759093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:42.759094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:42.759314 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:42.759415 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:42.759632 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 05:08:42.759734 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 05:08:42.760305 2368 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53970
I20260811 05:08:42.761173 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2449
I20260811 05:08:42.769068 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2614
I20260811 05:08:42.777834 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2783
I20260811 05:08:42.784886 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2329
2026-08-11T05:08:42Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2196 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2955](info): set up 2 sockets
Aug 11 05:08:42 dist-test-slave-7c35 krb5kdc[2955](info): commencing operation
krb5kdc: starting...
W20260811 05:08:43.856498 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:43 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:43.973156 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41067
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:41067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:44.077559 2970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:44.077857 2970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:44.077893 2970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:44.081774 2970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:44.081823 2970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:44.081842 2970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:44.081882 2970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:44.081915 2970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:44.086726 2970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:41067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:44.087689 2970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:44.088655 2970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:44.096521 2975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.096539 2978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.096590 2976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:44.096761 2970 server_base.cc:1034] running on GCE node
I20260811 05:08:44.097455 2970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:44.098496 2970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:44.099651 2970 hybrid_clock.cc:648] HybridClock initialized: now 1786424924099634 us; error 28 us; skew 500 ppm
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:44.102419 2970 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:44.103350 2970 webserver.cc:460] Webserver started at http://127.30.88.254:40281/ using document root <none> and password file <none>
I20260811 05:08:44.103890 2970 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:44.103950 2970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:44.104154 2970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:44.106009 2970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "84e046a936ae417fa3f72a4d202db525"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "84320b9691893c1dd5c515c4f92cd731"
server_key_iv: "43f0efc1f9cb7ac075f99602fc8ad641"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.106526 2970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "84e046a936ae417fa3f72a4d202db525"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "84320b9691893c1dd5c515c4f92cd731"
server_key_iv: "43f0efc1f9cb7ac075f99602fc8ad641"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.109941 2970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:44.112205 2986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.113312 2970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:44.113447 2970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "84e046a936ae417fa3f72a4d202db525"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "84320b9691893c1dd5c515c4f92cd731"
server_key_iv: "43f0efc1f9cb7ac075f99602fc8ad641"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.113562 2970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:44.135914 2970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:44.139012 2970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:44.139195 2970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:44.146101 2970 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:41067
I20260811 05:08:44.146128 3044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:41067 every 8 connection(s)
I20260811 05:08:44.147121 2970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:44.148746 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 2970
I20260811 05:08:44.148844 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:44.149175 31075 external_mini_cluster.cc:1428] Setting key ae1821bcbba31637ffef3feed306fd1b
I20260811 05:08:44.150457 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:44.157958 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525: Bootstrap starting.
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:44.160602 3045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:44.161340 3045 log.cc:824] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:44.163558 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525: No bootstrap required, opened a new log
I20260811 05:08:44.164760 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.150670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.30.88.254:41067)
0811 05:08:44.151154 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.151175 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.151221 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:08:44.152179 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.152204 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.152666 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.153040 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.153735 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.155451 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.156298 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.157111 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.157282 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.160083 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.160088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.160100 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.160124 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.161976 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.162536 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.162541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.162544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.162646 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.162950 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.162952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.162953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.163282 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.163413 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.163806 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.164103 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":176,"threads_started":1}
I20260811 05:08:44.167047 3045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } }
I20260811 05:08:44.167261 3045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:44.167348 3045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e046a936ae417fa3f72a4d202db525, State: Initialized, Role: FOLLOWER
I20260811 05:08:44.167738 3045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } }
I20260811 05:08:44.167891 3045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:44.167956 3045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:44.168082 3045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:44.168921 3045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } }
I20260811 05:08:44.169272 3045 leader_election.cc:302] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84e046a936ae417fa3f72a4d202db525; no voters:
I20260811 05:08:44.169559 3045 leader_election.cc:288] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:44.169641 3051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:44.169881 3051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [term 1 LEADER]: Becoming Leader. State: Replica: 84e046a936ae417fa3f72a4d202db525, State: Running, Role: LEADER
I20260811 05:08:44.170213 3051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } }
I20260811 05:08:44.170943 3045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:44.172048 3053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84e046a936ae417fa3f72a4d202db525. Latest consensus state: current_term: 1 leader_uuid: "84e046a936ae417fa3f72a4d202db525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } } }
I20260811 05:08:44.172196 3053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:44.172480 3052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84e046a936ae417fa3f72a4d202db525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e046a936ae417fa3f72a4d202db525" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41067 } } }
I20260811 05:08:44.172585 3052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:44.172675 3061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:44.175323 3061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:44.180363 3061 catalog_manager.cc:1347] Generated new cluster ID: c2839b99a2f1424b90c923b72db94a78
I20260811 05:08:44.180424 3061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:44.207274 3061 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:44.208194 3061 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:44.216741 3061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84e046a936ae417fa3f72a4d202db525: Generated new TSK 0
I20260811 05:08:44.217528 3061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:44.253753 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:44.370721 3076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:44.371006 3076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:44.371053 3076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:44.376219 3076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:44.376278 3076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:44.376349 3076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:44.382299 3076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:44.383384 3076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:44.384434 3076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:44.394435 3084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.394444 3082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.394444 3081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:44.394734 3076 server_base.cc:1034] running on GCE node
I20260811 05:08:44.395202 3076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:44.395804 3076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:44.397001 3076 hybrid_clock.cc:648] HybridClock initialized: now 1786424924396986 us; error 27 us; skew 500 ppm
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:44.400290 3076 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:44.401149 3076 webserver.cc:460] Webserver started at http://127.30.88.193:39807/ using document root <none> and password file <none>
I20260811 05:08:44.401733 3076 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:44.401794 3076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:44.401968 3076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:44.404186 3076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "b6528ea4006246d1bd60311a9c4c300a"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "005d37a58836c534f4c34a89b10753c3"
server_key_iv: "8ab89b020c50488bd8c42664a4f6d924"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.404806 3076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "b6528ea4006246d1bd60311a9c4c300a"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "005d37a58836c534f4c34a89b10753c3"
server_key_iv: "8ab89b020c50488bd8c42664a4f6d924"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.408746 3076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:08:44.411191 3094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.412432 3076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:08:44.412557 3076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "b6528ea4006246d1bd60311a9c4c300a"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "005d37a58836c534f4c34a89b10753c3"
server_key_iv: "8ab89b020c50488bd8c42664a4f6d924"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.412662 3076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:44.428331 3076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:44.431754 3076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:44.431974 3076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:44.432554 3076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:44.433413 3076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:44.433463 3076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.433537 3076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:44.433576 3076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.442456 3076 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:39681
I20260811 05:08:44.442489 3226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:39681 every 8 connection(s)
I20260811 05:08:44.443329 3076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:44.450613 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3076
I20260811 05:08:44.450739 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:44.451062 31075 external_mini_cluster.cc:1428] Setting key 2a771d8fa21cef1edee960a39b2d79e9
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:44.456810 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.445155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:44031 (local address 127.30.88.254:41067)
0811 05:08:44.445295 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.445298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.446117 (+ 819us) server_negotiation.cc:407] Connection header received
0811 05:08:44.447546 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.447549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.447608 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.447712 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.448754 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.449311 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.450457 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.450719 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.450848 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.453276 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.453279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.453281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.453303 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.454665 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.455275 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.455279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.455283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.455373 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.455681 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.455686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.455688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.455910 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.456004 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.456427 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.456587 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:08:44.457326 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.445539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41067 (local address 127.30.88.193:44031)
0811 05:08:44.445981 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:44.446021 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:44.447339 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:44.447876 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:44.447883 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:44.448203 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:44.448613 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.448625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.449436 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.449441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.450297 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.450307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.450925 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.450930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.451098 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.451690 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:44.451709 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:44.453110 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:44.454783 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.454792 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.454820 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.455183 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.455468 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.455470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.455471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.455587 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.456003 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:44.456008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:44.456196 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 05:08:44.456332 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:08:44.456600 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260811 05:08:44.458493 3227 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41067
I20260811 05:08:44.458726 3227 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:44.459300 3227 heartbeater.cc:511] Master 127.30.88.254:41067 requested a full tablet report, sending...
I20260811 05:08:44.460867 3005 ts_manager.cc:194] Registered new tserver with Master: b6528ea4006246d1bd60311a9c4c300a (127.30.88.193:39681)
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:44.461957 3005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:44031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:44.480242 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:44.584730 3232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:44.584981 3232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:44.585036 3232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:44.588923 3232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:44.588979 3232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:44.589030 3232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:44.594023 3232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:44.595026 3232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:44.595956 3232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:44.604902 3240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.604980 3238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.604897 3237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:44.605335 3232 server_base.cc:1034] running on GCE node
I20260811 05:08:44.605716 3232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:44.606299 3232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:44.607486 3232 hybrid_clock.cc:648] HybridClock initialized: now 1786424924607473 us; error 34 us; skew 500 ppm
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:44.610033 3232 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:44.610802 3232 webserver.cc:460] Webserver started at http://127.30.88.194:40527/ using document root <none> and password file <none>
I20260811 05:08:44.611348 3232 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:44.611454 3232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:44.611670 3232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:44.613639 3232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "e11cb53e8cfc4938afd377b4c23d6a21"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "408abefd106b8eaae7108a3c8a0ed801"
server_key_iv: "782db11b0f6f9b161de77d3a1073fc21"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.614243 3232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "e11cb53e8cfc4938afd377b4c23d6a21"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "408abefd106b8eaae7108a3c8a0ed801"
server_key_iv: "782db11b0f6f9b161de77d3a1073fc21"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.617905 3232 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:44.620249 3248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.621507 3232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:44.621636 3232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "e11cb53e8cfc4938afd377b4c23d6a21"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "408abefd106b8eaae7108a3c8a0ed801"
server_key_iv: "782db11b0f6f9b161de77d3a1073fc21"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.621755 3232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:44.648049 3232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:44.650741 3232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:44.650966 3232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:44.651544 3232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:44.652323 3232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:44.652390 3232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.652483 3232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:44.652525 3232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.660943 3232 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:35349
I20260811 05:08:44.660995 3383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:35349 every 8 connection(s)
I20260811 05:08:44.661773 3232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:44.666406 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3232
I20260811 05:08:44.666507 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:44.666790 31075 external_mini_cluster.cc:1428] Setting key 6aa094d73a41a480cd3aa016a024f22b
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:44.675879 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.663331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:49231 (local address 127.30.88.254:41067)
0811 05:08:44.663450 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.663453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.664204 (+ 751us) server_negotiation.cc:407] Connection header received
0811 05:08:44.665502 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.665505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.665558 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.665647 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.666900 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.667445 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.668899 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.669108 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.669186 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.672057 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.672100 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.672104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.672145 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.673799 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.674273 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.674280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.674283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.674384 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.674718 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.674723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.674727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.675037 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.675154 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.675569 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.675690 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:44.676502 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.663646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41067 (local address 127.30.88.194:49231)
0811 05:08:44.664062 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:44.664101 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:44.665312 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:44.665800 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:44.665806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:44.666090 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:44.666701 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.666721 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.667604 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.667607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.668781 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.668794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.669246 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.669252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.669462 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.670058 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:44.670105 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:44.671894 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:44.673907 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.673912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.673923 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.674164 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.674493 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.674495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.674496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.674618 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.675142 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:44.675147 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:44.675346 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 05:08:44.675489 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 05:08:44.675725 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260811 05:08:44.677537 3385 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41067
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:08:44.677752 3385 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:44.678223 3385 heartbeater.cc:511] Master 127.30.88.254:41067 requested a full tablet report, sending...
I20260811 05:08:44.679175 3005 ts_manager.cc:194] Registered new tserver with Master: e11cb53e8cfc4938afd377b4c23d6a21 (127.30.88.194:35349)
I20260811 05:08:44.679675 3005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:49231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:44.695881 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:44.800171 3391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:44.800391 3391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:44.800449 3391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:44.804602 3391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:44.804654 3391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:44.804751 3391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:44.809664 3391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:44.810622 3391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:44.811635 3391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:44.820948 3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.820973 3401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:44.820958 3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:44.821273 3391 server_base.cc:1034] running on GCE node
I20260811 05:08:44.821709 3391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:44.822294 3391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:44.823447 3391 hybrid_clock.cc:648] HybridClock initialized: now 1786424924823425 us; error 33 us; skew 500 ppm
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:44.826241 3391 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:44.827194 3391 webserver.cc:460] Webserver started at http://127.30.88.195:39173/ using document root <none> and password file <none>
I20260811 05:08:44.827731 3391 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:44.827791 3391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:44.828035 3391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:44.829933 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "9dae012abbde40cbb6636979622473dc"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "32437ff127704a4ef85eb6799098f35f"
server_key_iv: "44d1593faddd49dd7932ea358d8fd5d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.830534 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "9dae012abbde40cbb6636979622473dc"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "32437ff127704a4ef85eb6799098f35f"
server_key_iv: "44d1593faddd49dd7932ea358d8fd5d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.834163 3391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:44.836529 3409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.837666 3391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:44.837795 3391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "9dae012abbde40cbb6636979622473dc"
format_stamp: "Formatted at 2026-08-11 05:08:44 on dist-test-slave-7c35"
server_key: "32437ff127704a4ef85eb6799098f35f"
server_key_iv: "44d1593faddd49dd7932ea358d8fd5d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:44.837898 3391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:44.852180 3391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:44.855001 3391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:44.855185 3391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:44.855727 3391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:44.856514 3391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:44.856586 3391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.856657 3391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:44.856700 3391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:44.865526 3391 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:35125
I20260811 05:08:44.865567 3542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:35125 every 8 connection(s)
I20260811 05:08:44.866372 3391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:44.871742 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3391
I20260811 05:08:44.871881 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:44.872229 31075 external_mini_cluster.cc:1428] Setting key 186955db0d5a6064d2749c53bab2d975
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:44.880084 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.868097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:43103 (local address 127.30.88.254:41067)
0811 05:08:44.868231 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.868235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.869012 (+ 777us) server_negotiation.cc:407] Connection header received
0811 05:08:44.870464 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.870468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.870526 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.870612 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.871972 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.872654 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.873973 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.874227 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.874329 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.876899 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.876903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.876905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.876927 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.878124 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.878645 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.878648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.878649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.878717 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.879135 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.879138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.879139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.879305 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.879381 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.879865 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.879956 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:08:44.880952 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.868444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41067 (local address 127.30.88.195:43103)
0811 05:08:44.868870 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:44.868909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:44.870231 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:44.870792 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:44.870799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:44.871205 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:44.871776 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.871790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.872778 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.872782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.873857 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:44.873868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.874535 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.874539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:44.874693 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.875323 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:44.875339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:44.876751 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:44.878251 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.878256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.878266 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.878551 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.878799 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:44.878801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:44.878802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:44.879018 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:44.879407 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:44.879412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:44.879650 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 05:08:44.879807 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 05:08:44.880059 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":110,"threads_started":1}
I20260811 05:08:44.882028 3544 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41067
I20260811 05:08:44.882230 3544 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:44.882659 3544 heartbeater.cc:511] Master 127.30.88.254:41067 requested a full tablet report, sending...
I20260811 05:08:44.883700 3005 ts_manager.cc:194] Registered new tserver with Master: 9dae012abbde40cbb6636979622473dc (127.30.88.195:35125)
I20260811 05:08:44.884254 3005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:43103
I20260811 05:08:44.886269 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:44.913401 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.903984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.30.88.193:39681)
0811 05:08:44.904339 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.904349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.904371 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:08:44.904420 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.904425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.904610 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.904726 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.905352 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.906103 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.906833 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.907226 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.907308 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.909662 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.909673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.909685 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.909708 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.911656 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.912095 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.912100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.912101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.912218 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.912523 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.912525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.912526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.912741 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.912893 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.913129 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.913247 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":160,"threads_started":1}
Aug 11 05:08:44 dist-test-slave-7c35 krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:44.924358 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:44.916299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.30.88.254:41067)
0811 05:08:44.916476 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:44.916480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:44.916494 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:44.916638 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:44.916641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:44.916692 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:44.916759 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:44.917378 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.918111 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.918894 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:44.919123 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:44.919207 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:44.921459 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:44.921463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:44.921467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:44.921496 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:44.922695 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.923133 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.923136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.923138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.923219 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:44.923495 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:44.923498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:44.923500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:44.923717 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:44.923856 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:44.924088 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 05:08:44.924202 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 05:08:44.924790 3005 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40368
I20260811 05:08:44.925446 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3076
I20260811 05:08:44.934581 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3232
I20260811 05:08:44.942939 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3391
I20260811 05:08:44.950894 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 2970
2026-08-11T05:08:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2166 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:45 dist-test-slave-7c35 krb5kdc[3556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:45 dist-test-slave-7c35 krb5kdc[3556](info): set up 2 sockets
Aug 11 05:08:45 dist-test-slave-7c35 krb5kdc[3556](info): commencing operation
krb5kdc: starting...
W20260811 05:08:46.030774 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:46.143752 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37799
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:37799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:46.245908 3569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:46.246115 3569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:46.246169 3569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:46.249711 3569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:46.249756 3569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:46.249775 3569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:46.249791 3569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:46.249805 3569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:46.254515 3569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:37799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:46.255350 3569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:46.256145 3569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:46.263120 3577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:46.263125 3574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.263219 3569 server_base.cc:1034] running on GCE node
W20260811 05:08:46.263125 3575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.263947 3569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:46.264854 3569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:46.266006 3569 hybrid_clock.cc:648] HybridClock initialized: now 1786424926265989 us; error 27 us; skew 500 ppm
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:46.268826 3569 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:46.269438 3569 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:46.269500 3569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:46.269675 3569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:46.271658 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "04f76ddc2a3844f8865883225e8cb64d"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "eb60230b8f995ec3e3dfe67543b9fbca"
server_key_iv: "09cbd9f5565f2d6c90acd93411b08497"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.272151 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "04f76ddc2a3844f8865883225e8cb64d"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "eb60230b8f995ec3e3dfe67543b9fbca"
server_key_iv: "09cbd9f5565f2d6c90acd93411b08497"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.275493 3569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:46.277572 3583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.278776 3569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:46.278931 3569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "04f76ddc2a3844f8865883225e8cb64d"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "eb60230b8f995ec3e3dfe67543b9fbca"
server_key_iv: "09cbd9f5565f2d6c90acd93411b08497"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.279042 3569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:46.293185 3569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:46.295879 3569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:46.296075 3569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:46.302215 3569 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:37799
I20260811 05:08:46.302467 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:37799 every 8 connection(s)
I20260811 05:08:46.302819 3569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:46.305459 3651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:46.308833 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3569
I20260811 05:08:46.308961 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:46.309196 31075 external_mini_cluster.cc:1428] Setting key c14a0921a5b374e9c9f5cc5f6993d1e0
I20260811 05:08:46.310945 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: Bootstrap starting.
I20260811 05:08:46.313390 3651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:46.314139 3651 log.cc:824] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:46.316124 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: No bootstrap required, opened a new log
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:46.319991 3651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } }
I20260811 05:08:46.320233 3651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:46.320322 3651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f76ddc2a3844f8865883225e8cb64d, State: Initialized, Role: FOLLOWER
I20260811 05:08:46.320839 3651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } }
I20260811 05:08:46.321034 3651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:46.321115 3651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:46.321218 3651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:46.322254 3651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } }
I20260811 05:08:46.322616 3651 leader_election.cc:302] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f76ddc2a3844f8865883225e8cb64d; no voters:
I20260811 05:08:46.322928 3651 leader_election.cc:288] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:46.323051 3657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:46.323141 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.310496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.30.88.254:37799)
0811 05:08:46.310924 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:46.310950 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:46.310982 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:08:46.311600 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:46.311614 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:46.311890 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:46.312147 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:46.313128 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.314145 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.315264 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.315677 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.315779 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.318349 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:46.318353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:46.318363 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:46.318404 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:46.320022 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.320552 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.320558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.320564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.320714 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.320998 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.321002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.321004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.321532 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:46.321738 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:46.321935 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:08:46.322227 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":118,"threads_started":1}
I20260811 05:08:46.323261 3657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [term 1 LEADER]: Becoming Leader. State: Replica: 04f76ddc2a3844f8865883225e8cb64d, State: Running, Role: LEADER
I20260811 05:08:46.323551 3657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } }
I20260811 05:08:46.324082 3651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:46.325577 3658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04f76ddc2a3844f8865883225e8cb64d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } } }
I20260811 05:08:46.325709 3658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:46.326359 3659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f76ddc2a3844f8865883225e8cb64d. Latest consensus state: current_term: 1 leader_uuid: "04f76ddc2a3844f8865883225e8cb64d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f76ddc2a3844f8865883225e8cb64d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37799 } } }
I20260811 05:08:46.326437 3670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:46.326498 3659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:46.329241 3670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:46.334061 3670 catalog_manager.cc:1347] Generated new cluster ID: e647e4f03cd845a38d4747c6188e29d1
I20260811 05:08:46.334122 3670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:46.347317 3670 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:46.348338 3670 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:46.355906 3670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: Generated new TSK 0
I20260811 05:08:46.356531 3670 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:46.388240 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:46.498186 3681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:46.498430 3681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:46.498490 3681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:46.502127 3681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:46.502177 3681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:46.502223 3681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:46.506997 3681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:46.507882 3681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:46.508733 3681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:46.517544 3690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:46.517602 3686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.517681 3681 server_base.cc:1034] running on GCE node
W20260811 05:08:46.517591 3687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.518164 3681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:46.518700 3681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:46.519852 3681 hybrid_clock.cc:648] HybridClock initialized: now 1786424926519835 us; error 34 us; skew 500 ppm
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:46.522617 3681 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:46.523511 3681 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:46.523602 3681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:46.523810 3681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:46.525815 3681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "62572cfbe858487d8d66191d412de307"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "cab9075c36e5c07eeae0436adcf624af"
server_key_iv: "7f345b5ef673313228171d9ba7735e00"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.526338 3681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "62572cfbe858487d8d66191d412de307"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "cab9075c36e5c07eeae0436adcf624af"
server_key_iv: "7f345b5ef673313228171d9ba7735e00"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.530031 3681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:46.532491 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.533699 3681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:46.533820 3681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "62572cfbe858487d8d66191d412de307"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "cab9075c36e5c07eeae0436adcf624af"
server_key_iv: "7f345b5ef673313228171d9ba7735e00"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.533944 3681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:46.544817 3681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:46.547494 3681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:46.547674 3681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:46.548207 3681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:46.549017 3681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:46.549085 3681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.549153 3681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:46.549202 3681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.557937 3681 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:38679
I20260811 05:08:46.557976 3835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:38679 every 8 connection(s)
I20260811 05:08:46.558513 3681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:46.564231 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3681
I20260811 05:08:46.564375 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:46.564679 31075 external_mini_cluster.cc:1428] Setting key e0932d761ccfea54c0ca6940f6dc0e85
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:46.572171 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.560104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:49223 (local address 127.30.88.254:37799)
0811 05:08:46.560239 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:46.560244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:46.561072 (+ 828us) server_negotiation.cc:407] Connection header received
0811 05:08:46.562318 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:46.562321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:46.562365 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:46.562433 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:46.563562 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.564256 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.565618 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.565887 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.565989 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.568474 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:46.568480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:46.568483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:46.568515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:46.569990 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.570487 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.570494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.570499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.570607 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.570994 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.571001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.571005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.571294 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:46.571414 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:46.571860 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 05:08:46.571994 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:08:46.572882 3841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.560463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37799 (local address 127.30.88.193:49223)
0811 05:08:46.560923 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:46.560963 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:46.562136 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:46.562571 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:46.562577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:46.562989 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:46.563432 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.563446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.564421 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.564427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.565492 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.565502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.566045 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.566050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.566227 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.566777 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:46.566801 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:46.568312 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:46.570105 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.570109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.570137 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.570380 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.570715 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.570717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.570719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.570844 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.571417 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:46.571421 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:46.571633 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 05:08:46.571806 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 05:08:46.572091 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260811 05:08:46.574250 3839 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37799
I20260811 05:08:46.574502 3839 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:46.575003 3839 heartbeater.cc:511] Master 127.30.88.254:37799 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:46.576561 3606 ts_manager.cc:194] Registered new tserver with Master: 62572cfbe858487d8d66191d412de307 (127.30.88.193:38679)
I20260811 05:08:46.577715 3606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:49223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:46.594578 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:46.705255 3845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:46.705480 3845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:46.705523 3845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:46.709343 3845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:46.709393 3845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:46.709517 3845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:46.715142 3845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:46.715984 3845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:46.716856 3845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:46.726233 3850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:46.726258 3851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.726382 3845 server_base.cc:1034] running on GCE node
W20260811 05:08:46.726537 3853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.726994 3845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:46.727644 3845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:46.728833 3845 hybrid_clock.cc:648] HybridClock initialized: now 1786424926728812 us; error 43 us; skew 500 ppm
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:46.731698 3845 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:46.732461 3845 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:46.732558 3845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:46.732761 3845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:46.734679 3845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "d85e0061be1f43e1a9674c953187cbec"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "9740c824b82bcad503a5666df6d022a3"
server_key_iv: "52cdd12f5d6cd90ea7553ef0018509fa"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.735308 3845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "d85e0061be1f43e1a9674c953187cbec"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "9740c824b82bcad503a5666df6d022a3"
server_key_iv: "52cdd12f5d6cd90ea7553ef0018509fa"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.739166 3845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:08:46.741595 3862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.742722 3845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:46.742897 3845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "d85e0061be1f43e1a9674c953187cbec"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "9740c824b82bcad503a5666df6d022a3"
server_key_iv: "52cdd12f5d6cd90ea7553ef0018509fa"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.743045 3845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:46.756444 3845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:46.759200 3845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:46.759397 3845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:46.759968 3845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:46.760774 3845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:46.760847 3845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.760918 3845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:46.760969 3845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.769513 3845 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:34515
I20260811 05:08:46.769541 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:34515 every 8 connection(s)
I20260811 05:08:46.770066 3845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:46.770453 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 3845
I20260811 05:08:46.770594 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:46.770984 31075 external_mini_cluster.cc:1428] Setting key bd6ae20e9201e0ff298f4c47dcfa0889
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:08:46.785034 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.772091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:39921 (local address 127.30.88.254:37799)
0811 05:08:46.772220 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:46.772226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:46.773122 (+ 896us) server_negotiation.cc:407] Connection header received
0811 05:08:46.774453 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:46.774456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:46.774510 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:46.774591 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:46.775980 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.776509 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.778344 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.778578 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.778666 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.781464 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:46.781468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:46.781473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:46.781507 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:46.782986 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.783530 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.783534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.783535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.783615 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.783968 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.783973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.783975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.784153 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:46.784258 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:46.784745 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 05:08:46.784863 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:08:46.785737 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.772513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37799 (local address 127.30.88.194:39921)
0811 05:08:46.772988 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:46.773027 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:46.774245 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:46.774889 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:46.774900 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:46.775280 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:46.775816 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.775839 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.776638 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.776644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.778196 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.778212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.778744 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.778750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.779032 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.779631 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:46.779653 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:46.781302 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:46.783116 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.783121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.783134 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.783425 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.783706 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.783711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.783715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.783870 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.784270 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:46.784278 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:46.784507 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 05:08:46.784684 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 05:08:46.784973 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":144,"threads_started":1}
I20260811 05:08:46.787046 3996 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37799
I20260811 05:08:46.787274 3996 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:46.787685 3996 heartbeater.cc:511] Master 127.30.88.254:37799 requested a full tablet report, sending...
I20260811 05:08:46.788830 3606 ts_manager.cc:194] Registered new tserver with Master: d85e0061be1f43e1a9674c953187cbec (127.30.88.194:34515)
I20260811 05:08:46.789410 3606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:39921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:46.801288 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:46.908823 4003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:46.909052 4003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:46.909124 4003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:46.912686 4003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:46.912729 4003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:46.912777 4003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:46.917603 4003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:46.918454 4003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:46.919404 4003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:46.928107 4008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.928328 4003 server_base.cc:1034] running on GCE node
W20260811 05:08:46.928107 4009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:46.928186 4011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:46.928886 4003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:46.929370 4003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:46.930543 4003 hybrid_clock.cc:648] HybridClock initialized: now 1786424926930527 us; error 27 us; skew 500 ppm
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:46.933189 4003 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:46.933813 4003 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:46.933890 4003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:46.934052 4003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:46.935894 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "f883341bbeb54ba59e80c592523941ac"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "c0050844590a723ac3ac1e158f1cb751"
server_key_iv: "87bdaa085406f0ec272047cb60c090ae"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.936409 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "f883341bbeb54ba59e80c592523941ac"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "c0050844590a723ac3ac1e158f1cb751"
server_key_iv: "87bdaa085406f0ec272047cb60c090ae"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.939812 4003 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:08:46.942081 4017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.943188 4003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:08:46.943300 4003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f883341bbeb54ba59e80c592523941ac"
format_stamp: "Formatted at 2026-08-11 05:08:46 on dist-test-slave-7c35"
server_key: "c0050844590a723ac3ac1e158f1cb751"
server_key_iv: "87bdaa085406f0ec272047cb60c090ae"
server_key_version: "encryptionkey@0"
I20260811 05:08:46.943395 4003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:46.960790 4003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:46.963534 4003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:46.963727 4003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:46.964278 4003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:46.965066 4003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:46.965111 4003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.965174 4003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:46.965207 4003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:46.973398 4003 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:43191
I20260811 05:08:46.973506 4154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:43191 every 8 connection(s)
I20260811 05:08:46.973997 4003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:46.977084 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4003
I20260811 05:08:46.977171 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:46.977442 31075 external_mini_cluster.cc:1428] Setting key ea2f226e73205810e986343fa5369d7b
Aug 11 05:08:46 dist-test-slave-7c35 krb5kdc[3556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:46.986439 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.975747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:42371 (local address 127.30.88.254:37799)
0811 05:08:46.975881 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:46.975885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:46.976616 (+ 731us) server_negotiation.cc:407] Connection header received
0811 05:08:46.977991 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:46.977995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:46.978042 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:46.978147 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:46.979436 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.979936 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.980927 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.981118 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.981187 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.983506 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:46.983508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:46.983510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:46.983531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:46.984795 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.985239 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.985242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.985243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.985314 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.985613 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.985615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.985616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.985770 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:46.985841 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:46.986239 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 05:08:46.986327 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:08:46.987176 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.976068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37799 (local address 127.30.88.195:42371)
0811 05:08:46.976483 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:46.976520 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:46.977766 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:46.978340 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:46.978348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:46.978732 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:46.979294 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.979315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.980046 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.980048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.980829 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:46.980838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.981231 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.981233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:46.981364 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.981921 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:46.981937 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:46.983363 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:46.984894 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.984899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.984909 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.985156 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.985390 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:46.985392 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:46.985393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:46.985528 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:46.985852 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:46.985856 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:46.986051 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 05:08:46.986192 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 05:08:46.986405 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260811 05:08:46.988315 4155 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37799
I20260811 05:08:46.988544 4155 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:46.988956 4155 heartbeater.cc:511] Master 127.30.88.254:37799 requested a full tablet report, sending...
I20260811 05:08:46.990146 3606 ts_manager.cc:194] Registered new tserver with Master: f883341bbeb54ba59e80c592523941ac (127.30.88.195:43191)
I20260811 05:08:46.990931 3606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:42371
I20260811 05:08:46.991235 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:08:46.999816 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:46.993165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.30.88.254:37799)
0811 05:08:46.993275 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:46.993279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:46.993479 (+ 200us) server_negotiation.cc:407] Connection header received
0811 05:08:46.993569 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:46.993572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:46.993607 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:46.993678 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:46.994305 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.995117 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.995961 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:46.996146 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:46.996215 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:46.997285 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:46.997287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:46.997288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:46.997303 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:46.998297 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.998701 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.998704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.998705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.998788 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:46.999121 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:46.999124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:46.999125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:46.999325 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:46.999448 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:46.999616 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 05:08:46.999701 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:08:47.002318 3606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:47.005354 3606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:47.019802 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.013891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.30.88.193:38679)
0811 05:08:47.014367 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.014382 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.014783 (+ 401us) server_negotiation.cc:407] Connection header received
0811 05:08:47.015090 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.015127 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.015333 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.015472 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:47.016097 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.016863 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.018338 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.018983 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.019053 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.019161 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.019553 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.019649 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":119,"threads_started":1}
I20260811 05:08:47.020079 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.014483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.30.88.194:34515)
0811 05:08:47.014901 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.014914 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.015203 (+ 289us) server_negotiation.cc:407] Connection header received
0811 05:08:47.015427 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.015432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.015575 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.015686 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:47.016189 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.016927 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.018338 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.019184 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.019263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.019406 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.019858 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.019957 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":109,"threads_started":1}
I20260811 05:08:47.019778 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.013676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:38679 (local address 127.0.0.1:39716)
0811 05:08:47.014628 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.014683 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.014885 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.015694 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.015701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.015741 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:47.015961 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.015970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.017018 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.017023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.018193 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.018205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.019245 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.019248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.019342 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.019393 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.019508 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.019623 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":111,"spinlock_wait_cycles":5376,"thread_start_us":74,"threads_started":1}
I20260811 05:08:47.020699 4170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.013869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:43191 (local address 127.0.0.1:46816)
0811 05:08:47.014628 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.014683 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.014820 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.015694 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.015701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.015743 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:47.015961 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.015970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.018192 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.018195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.019287 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.019297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.020361 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.020365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.020462 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.020479 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.020548 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.020594 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":31,"spinlock_wait_cycles":6656,"thread_start_us":88,"threads_started":1}
I20260811 05:08:47.021137 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.014472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.30.88.195:43191)
0811 05:08:47.015270 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.015281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.015300 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:08:47.015388 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.015394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.015569 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.015701 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:47.017004 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.018052 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.019443 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.020203 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.020313 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.020474 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.020598 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.020707 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":105,"threads_started":1}
I20260811 05:08:47.021307 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.014216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34515 (local address 127.0.0.1:32802)
0811 05:08:47.015098 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.015113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.015275 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.015894 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.015896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.015912 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:47.016001 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.016005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.017104 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.017107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.018220 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.018228 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.019515 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.019518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.019613 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.019634 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.019697 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.019760 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":3,"thread_start_us":124,"threads_started":1}
I20260811 05:08:47.023521 3756 tablet_service.cc:1467] Processing CreateTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 (DEFAULT_TABLE table=test-table [id=3e8e00fa332b4f2bad6db16795cde565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.023958 4076 tablet_service.cc:1467] Processing CreateTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 (DEFAULT_TABLE table=test-table [id=3e8e00fa332b4f2bad6db16795cde565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.024072 3914 tablet_service.cc:1467] Processing CreateTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 (DEFAULT_TABLE table=test-table [id=3e8e00fa332b4f2bad6db16795cde565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.024722 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a739c5ef9b1940bc83ddcf7462a9f9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.025056 3914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a739c5ef9b1940bc83ddcf7462a9f9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.025156 4076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a739c5ef9b1940bc83ddcf7462a9f9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.031528 4177 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Bootstrap starting.
I20260811 05:08:47.032739 4180 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Bootstrap starting.
I20260811 05:08:47.033044 4179 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Bootstrap starting.
I20260811 05:08:47.034621 4180 tablet_bootstrap.cc:654] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.034960 4177 tablet_bootstrap.cc:654] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.035064 4179 tablet_bootstrap.cc:654] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.035919 4180 log.cc:824] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.036440 4177 log.cc:824] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.036440 4179 log.cc:824] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.038167 4180 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: No bootstrap required, opened a new log
I20260811 05:08:47.038445 4179 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: No bootstrap required, opened a new log
I20260811 05:08:47.038444 4180 ts_tablet_manager.cc:1397] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 05:08:47.038583 4177 tablet_bootstrap.cc:492] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: No bootstrap required, opened a new log
I20260811 05:08:47.038645 4179 ts_tablet_manager.cc:1397] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:08:47.038798 4177 ts_tablet_manager.cc:1397] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:08:47.043149 4177 raft_consensus.cc:348] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.043411 4177 raft_consensus.cc:374] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.043478 4177 raft_consensus.cc:729] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f883341bbeb54ba59e80c592523941ac, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.043406 4180 raft_consensus.cc:348] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.043406 4179 raft_consensus.cc:348] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.043663 4180 raft_consensus.cc:374] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.043663 4179 raft_consensus.cc:374] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.043720 4179 raft_consensus.cc:729] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85e0061be1f43e1a9674c953187cbec, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.043720 4180 raft_consensus.cc:729] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62572cfbe858487d8d66191d412de307, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.044018 4177 consensus_queue.cc:260] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.044147 4179 consensus_queue.cc:260] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.044147 4180 consensus_queue.cc:260] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.045073 4180 ts_tablet_manager.cc:1428] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:08:47.045064 4179 ts_tablet_manager.cc:1428] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 05:08:47.045061 4177 ts_tablet_manager.cc:1428] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:08:47.045449 3839 heartbeater.cc:503] Master 127.30.88.254:37799 was elected leader, sending a full tablet report...
I20260811 05:08:47.045529 4155 heartbeater.cc:503] Master 127.30.88.254:37799 was elected leader, sending a full tablet report...
I20260811 05:08:47.046468 3996 heartbeater.cc:503] Master 127.30.88.254:37799 was elected leader, sending a full tablet report...
W20260811 05:08:47.059577 3840 tablet.cc:2369] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:47.073479 4184 raft_consensus.cc:482] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:47.073732 4184 raft_consensus.cc:504] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.074949 4184 leader_election.cc:288] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62572cfbe858487d8d66191d412de307 (127.30.88.193:38679), d85e0061be1f43e1a9674c953187cbec (127.30.88.194:34515)
I20260811 05:08:47.079598 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.075148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:38679 (local address 127.30.88.195:45683)
0811 05:08:47.075323 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.075338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.075445 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.075753 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.075756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.075774 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:47.075923 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.075931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.076952 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.076954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.078217 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.078229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.079114 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.079117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.079249 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.079276 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.079366 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.079451 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 05:08:47.079631 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.075226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:45683 (local address 127.30.88.193:38679)
0811 05:08:47.075338 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.075343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.075408 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:08:47.075585 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.075588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.075639 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.075728 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:47.076045 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.076824 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.078351 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.078978 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.079098 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.079150 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.079392 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.079478 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:08:47.079659 4189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.075206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34515 (local address 127.30.88.195:41945)
0811 05:08:47.075669 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.075679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.075854 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.076156 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.076159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.076175 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:47.076320 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.076330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.077209 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.077213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.078391 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:47.078399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.079267 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.079271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:47.079380 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.079399 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.079483 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.079545 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":161,"threads_started":1}
I20260811 05:08:47.079689 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.075343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:41945 (local address 127.30.88.194:34515)
0811 05:08:47.075514 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.075521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.075780 (+ 259us) server_negotiation.cc:407] Connection header received
0811 05:08:47.075984 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.075995 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.076051 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.076128 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:47.076435 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.077086 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.078506 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.079140 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.079248 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.079282 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.079501 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.079573 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 05:08:47.080322 3783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a739c5ef9b1940bc83ddcf7462a9f9a7" candidate_uuid: "f883341bbeb54ba59e80c592523941ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62572cfbe858487d8d66191d412de307" is_pre_election: true
I20260811 05:08:47.080322 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a739c5ef9b1940bc83ddcf7462a9f9a7" candidate_uuid: "f883341bbeb54ba59e80c592523941ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85e0061be1f43e1a9674c953187cbec" is_pre_election: true
I20260811 05:08:47.080633 3783 raft_consensus.cc:2454] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f883341bbeb54ba59e80c592523941ac in term 0.
I20260811 05:08:47.080633 3936 raft_consensus.cc:2454] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f883341bbeb54ba59e80c592523941ac in term 0.
I20260811 05:08:47.081151 4020 leader_election.cc:302] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d85e0061be1f43e1a9674c953187cbec, f883341bbeb54ba59e80c592523941ac; no voters:
I20260811 05:08:47.081418 4184 raft_consensus.cc:2781] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:47.081518 4184 raft_consensus.cc:482] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:47.081579 4184 raft_consensus.cc:3037] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:47.082803 4184 raft_consensus.cc:504] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.083201 4184 leader_election.cc:288] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [CANDIDATE]: Term 1 election: Requested vote from peers 62572cfbe858487d8d66191d412de307 (127.30.88.193:38679), d85e0061be1f43e1a9674c953187cbec (127.30.88.194:34515)
I20260811 05:08:47.083539 3783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a739c5ef9b1940bc83ddcf7462a9f9a7" candidate_uuid: "f883341bbeb54ba59e80c592523941ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62572cfbe858487d8d66191d412de307"
I20260811 05:08:47.083668 3783 raft_consensus.cc:3037] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:47.083750 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a739c5ef9b1940bc83ddcf7462a9f9a7" candidate_uuid: "f883341bbeb54ba59e80c592523941ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85e0061be1f43e1a9674c953187cbec"
I20260811 05:08:47.083876 3936 raft_consensus.cc:3037] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:47.084993 3783 raft_consensus.cc:2454] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f883341bbeb54ba59e80c592523941ac in term 1.
I20260811 05:08:47.085245 3936 raft_consensus.cc:2454] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f883341bbeb54ba59e80c592523941ac in term 1.
I20260811 05:08:47.085340 4018 leader_election.cc:302] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62572cfbe858487d8d66191d412de307, f883341bbeb54ba59e80c592523941ac; no voters:
I20260811 05:08:47.085559 4184 raft_consensus.cc:2781] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:47.085815 4184 raft_consensus.cc:686] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 1 LEADER]: Becoming Leader. State: Replica: f883341bbeb54ba59e80c592523941ac, State: Running, Role: LEADER
I20260811 05:08:47.086154 4184 consensus_queue.cc:237] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } }
I20260811 05:08:47.089257 3605 catalog_manager.cc:5495] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac reported cstate change: term changed from 0 to 1, leader changed from <none> to f883341bbeb54ba59e80c592523941ac (127.30.88.195). New cstate: current_term: 1 leader_uuid: "f883341bbeb54ba59e80c592523941ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f883341bbeb54ba59e80c592523941ac" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:47.099462 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.096183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.30.88.195:43191)
0811 05:08:47.096298 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.096301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.096316 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:08:47.096465 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.096467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.096511 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.096591 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:47.096802 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.097286 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.098134 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:47.098341 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:47.098425 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:47.098685 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:47.099153 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:47.099243 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.099310 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.099354 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:47.104758 3936 raft_consensus.cc:1261] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 1 FOLLOWER]: Refusing update from remote peer f883341bbeb54ba59e80c592523941ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:47.104761 3783 raft_consensus.cc:1261] T a739c5ef9b1940bc83ddcf7462a9f9a7 P 62572cfbe858487d8d66191d412de307 [term 1 FOLLOWER]: Refusing update from remote peer f883341bbeb54ba59e80c592523941ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:47.105437 4184 consensus_queue.cc:1035] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "62572cfbe858487d8d66191d412de307" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 38679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:47.105613 4190 consensus_queue.cc:1035] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "d85e0061be1f43e1a9674c953187cbec" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:47.115875 4195 mvcc.cc:205] Tried to move back new op lower bound from 7317196501414940672 to 7317196501347385344. Current Snapshot: MvccSnapshot[applied={T|T < 7317196501414940672}]
I20260811 05:08:47.120561 3605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:47.120729 3605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:47.123350 3605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f76ddc2a3844f8865883225e8cb64d: Sending DeleteTablet for 3 replicas of tablet a739c5ef9b1940bc83ddcf7462a9f9a7
I20260811 05:08:47.124091 4076 tablet_service.cc:1514] Processing DeleteTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:47 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:46816
I20260811 05:08:47.124249 3756 tablet_service.cc:1514] Processing DeleteTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:47 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:39716
I20260811 05:08:47.124565 4203 tablet_replica.cc:330] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: stopping tablet replica
I20260811 05:08:47.124735 3914 tablet_service.cc:1514] Processing DeleteTablet for tablet a739c5ef9b1940bc83ddcf7462a9f9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:47 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:32802
I20260811 05:08:47.124971 4203 raft_consensus.cc:2229] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:47.124979 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3681
I20260811 05:08:47.125341 4203 raft_consensus.cc:2258] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:47.127722 4203 ts_tablet_manager.cc:1905] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:47.129207 4205 tablet_replica.cc:330] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: stopping tablet replica
I20260811 05:08:47.129522 4205 raft_consensus.cc:2229] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:47.129824 4205 raft_consensus.cc:2258] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:47.130831 4203 ts_tablet_manager.cc:1918] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:47.131013 4203 log.cc:1196] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/a739c5ef9b1940bc83ddcf7462a9f9a7
I20260811 05:08:47.131419 4205 ts_tablet_manager.cc:1905] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:47.131419 4203 ts_tablet_manager.cc:1939] T a739c5ef9b1940bc83ddcf7462a9f9a7 P f883341bbeb54ba59e80c592523941ac: Deleting consensus metadata
I20260811 05:08:47.132671 3589 catalog_manager.cc:4841] TS f883341bbeb54ba59e80c592523941ac (127.30.88.195:43191): tablet a739c5ef9b1940bc83ddcf7462a9f9a7 (table test-table [id=3e8e00fa332b4f2bad6db16795cde565]) successfully deleted
I20260811 05:08:47.134799 4205 ts_tablet_manager.cc:1918] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:47.134919 4205 log.cc:1196] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/a739c5ef9b1940bc83ddcf7462a9f9a7
I20260811 05:08:47.135239 4205 ts_tablet_manager.cc:1939] T a739c5ef9b1940bc83ddcf7462a9f9a7 P d85e0061be1f43e1a9674c953187cbec: Deleting consensus metadata
I20260811 05:08:47.136365 3588 catalog_manager.cc:4841] TS d85e0061be1f43e1a9674c953187cbec (127.30.88.194:34515): tablet a739c5ef9b1940bc83ddcf7462a9f9a7 (table test-table [id=3e8e00fa332b4f2bad6db16795cde565]) successfully deleted
W20260811 05:08:47.138370 3584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:38679 (error 108)
I20260811 05:08:47.138543 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3845
I20260811 05:08:47.143688 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.139858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:38679 (local address 127.0.0.1:39724)
0811 05:08:47.143499 (+ 3641us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.143585 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:38679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3533}
W20260811 05:08:47.143858 3584 catalog_manager.cc:4584] TS 62572cfbe858487d8d66191d412de307 (127.30.88.193:38679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a739c5ef9b1940bc83ddcf7462a9f9a7: Network error: Client connection negotiation failed: client connection to 127.30.88.193:38679: connect: Connection refused (error 111)
I20260811 05:08:47.149940 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4003
I20260811 05:08:47.156914 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 3569
2026-08-11T05:08:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2205 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T05:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:47Z Disabled control of system clock
I20260811 05:08:47.189347 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39073
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:41787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:08:47.298316 4211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:47.298556 4211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:47.298591 4211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:47.301977 4211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:47.302022 4211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:47.302039 4211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:47.302057 4211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:47.302073 4211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:47.306453 4211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:39073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:47.307284 4211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:47.308430 4211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:47.316103 4218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.316154 4221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:47.316192 4211 server_base.cc:1034] running on GCE node
W20260811 05:08:47.316169 4217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:47.317073 4211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:47.318037 4211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:47.319211 4211 hybrid_clock.cc:648] HybridClock initialized: now 1786424927319184 us; error 39 us; skew 500 ppm
I20260811 05:08:47.325030 4211 webserver.cc:460] Webserver started at http://127.30.88.254:41735/ using document root <none> and password file <none>
I20260811 05:08:47.325608 4211 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:47.325672 4211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:47.325918 4211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:47.328056 4211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "b3c0fdc11db53ba201e33aa5eba9f0cf"
server_key_iv: "b356524e84e03f31457b626492d4591d"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.328665 4211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "b3c0fdc11db53ba201e33aa5eba9f0cf"
server_key_iv: "b356524e84e03f31457b626492d4591d"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.332475 4211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 05:08:47.334933 4229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.336100 4211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:08:47.336210 4211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "b3c0fdc11db53ba201e33aa5eba9f0cf"
server_key_iv: "b356524e84e03f31457b626492d4591d"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.336290 4211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:47.340600 4211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:47.341104 4211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:47.341230 4211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:47.347764 4211 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:39073
I20260811 05:08:47.347784 4290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:39073 every 8 connection(s)
I20260811 05:08:47.348551 4211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:47.351583 4291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.354790 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4211
I20260811 05:08:47.354933 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:47.355237 31075 external_mini_cluster.cc:1428] Setting key 99ead7eb379f11882bc9108fc183dae5
I20260811 05:08:47.357313 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: Bootstrap starting.
I20260811 05:08:47.359514 4291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.360277 4291 log.cc:824] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.360589 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.356631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.30.88.254:39073)
0811 05:08:47.357252 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.357267 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.357302 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:08:47.357933 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.357948 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.358324 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.358636 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.359380 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.359386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.359405 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.359414 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.359465 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.359480 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.359589 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.359686 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.359912 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":107,"threads_started":1}
I20260811 05:08:47.362243 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: No bootstrap required, opened a new log
I20260811 05:08:47.365698 4291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } }
I20260811 05:08:47.365914 4291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.366008 4291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cca95982a4a4106bbf64c7a9ef5d3f0, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.366364 4291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } }
I20260811 05:08:47.366503 4291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:47.366583 4291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:47.366678 4291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:47.367503 4291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } }
I20260811 05:08:47.367834 4291 leader_election.cc:302] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cca95982a4a4106bbf64c7a9ef5d3f0; no voters:
I20260811 05:08:47.368173 4291 leader_election.cc:288] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:47.368235 4296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:47.368453 4296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [term 1 LEADER]: Becoming Leader. State: Replica: 7cca95982a4a4106bbf64c7a9ef5d3f0, State: Running, Role: LEADER
I20260811 05:08:47.368719 4296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } }
I20260811 05:08:47.369288 4291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:47.370223 4297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cca95982a4a4106bbf64c7a9ef5d3f0. Latest consensus state: current_term: 1 leader_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } } }
I20260811 05:08:47.370370 4297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:47.370654 4298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cca95982a4a4106bbf64c7a9ef5d3f0" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39073 } } }
I20260811 05:08:47.370754 4298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:47.370838 4307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:47.373644 4307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:47.380163 4307 catalog_manager.cc:1347] Generated new cluster ID: 3753be7c40e6468db52b34a9ab75abb0
I20260811 05:08:47.380224 4307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:47.386322 4307 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:47.386582 4307 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:47.395543 4307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: Generated new TSK 0
I20260811 05:08:47.396190 4307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:08:47.398185 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--builtin_ntp_servers=127.30.88.212:41787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:08:47.508256 4320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:47.508448 4320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:47.508479 4320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:47.512084 4320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:47.512125 4320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:47.512171 4320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:47.516480 4320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:47.517323 4320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:47.518501 4320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:47.527264 4331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.527284 4327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.527284 4329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:47.527422 4320 server_base.cc:1034] running on GCE node
I20260811 05:08:47.527885 4320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:47.528391 4320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:47.529536 4320 hybrid_clock.cc:648] HybridClock initialized: now 1786424927529511 us; error 37 us; skew 500 ppm
I20260811 05:08:47.531334 4320 webserver.cc:460] Webserver started at http://127.30.88.193:38665/ using document root <none> and password file <none>
I20260811 05:08:47.531898 4320 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:47.531966 4320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:47.532204 4320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:47.534058 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "06de5af2b20149269720f5364de2e178"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "3ec71376978d84d7d36786ac681592ee"
server_key_iv: "86519708577c93484567c86a6c843962"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.534588 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "06de5af2b20149269720f5364de2e178"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "3ec71376978d84d7d36786ac681592ee"
server_key_iv: "86519708577c93484567c86a6c843962"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.537969 4320 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:47.540263 4337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.541313 4320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:47.541441 4320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "06de5af2b20149269720f5364de2e178"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "3ec71376978d84d7d36786ac681592ee"
server_key_iv: "86519708577c93484567c86a6c843962"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.541570 4320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:47.545750 4320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:47.546265 4320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:47.546432 4320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:47.546959 4320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:47.547750 4320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:47.547813 4320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.547885 4320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:47.547920 4320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.555922 4320 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:45789
I20260811 05:08:47.555945 4477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:45789 every 8 connection(s)
I20260811 05:08:47.556840 4320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:47.562161 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.558538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:59535 (local address 127.30.88.254:39073)
0811 05:08:47.558699 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.558704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.559663 (+ 959us) server_negotiation.cc:407] Connection header received
0811 05:08:47.560847 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.560849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.560882 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.560946 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.561792 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.561794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.561796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.561806 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.561824 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.561831 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.561894 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.562016 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.562049 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:08:47.562906 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.558929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39073 (local address 127.30.88.193:59535)
0811 05:08:47.559523 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.559558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.560686 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.561086 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.561092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.561164 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.561583 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.561584 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.561617 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.561673 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.561897 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.561903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.561906 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.562012 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.562183 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260811 05:08:47.564141 4478 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39073
I20260811 05:08:47.564388 4478 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:47.564837 4478 heartbeater.cc:511] Master 127.30.88.254:39073 requested a full tablet report, sending...
I20260811 05:08:47.566144 4248 ts_manager.cc:194] Registered new tserver with Master: 06de5af2b20149269720f5364de2e178 (127.30.88.193:45789)
I20260811 05:08:47.566809 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4320
I20260811 05:08:47.566949 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:47.567202 31075 external_mini_cluster.cc:1428] Setting key 14ed395cbda7aefdf94dac86423fb8c4
I20260811 05:08:47.568779 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--builtin_ntp_servers=127.30.88.212:41787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:08:47.678659 4482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:47.678942 4482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:47.679023 4482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:47.682760 4482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:47.682828 4482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:47.682933 4482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:47.687615 4482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:47.688539 4482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:47.689754 4482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:47.698480 4488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.698472 4487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.698505 4490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:47.698766 4482 server_base.cc:1034] running on GCE node
I20260811 05:08:47.699157 4482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:47.699730 4482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:47.700888 4482 hybrid_clock.cc:648] HybridClock initialized: now 1786424927700871 us; error 30 us; skew 500 ppm
I20260811 05:08:47.702423 4482 webserver.cc:460] Webserver started at http://127.30.88.194:41837/ using document root <none> and password file <none>
I20260811 05:08:47.702986 4482 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:47.703047 4482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:47.703251 4482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:47.705139 4482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "3b9b407e022d4c13b0ad8ef2072eadee"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "325812e90176580f74055bbee17a30b7"
server_key_iv: "c003042e512230895f9e11c2e4707d95"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.705689 4482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "3b9b407e022d4c13b0ad8ef2072eadee"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "325812e90176580f74055bbee17a30b7"
server_key_iv: "c003042e512230895f9e11c2e4707d95"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.709035 4482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:47.711401 4497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.712483 4482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:47.712606 4482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "3b9b407e022d4c13b0ad8ef2072eadee"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "325812e90176580f74055bbee17a30b7"
server_key_iv: "c003042e512230895f9e11c2e4707d95"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.712729 4482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:47.717126 4482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:47.717710 4482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:47.717886 4482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:47.718379 4482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:47.719283 4482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:47.719347 4482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.719410 4482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:47.719452 4482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.727731 4482 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:40071
I20260811 05:08:47.727767 4631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:40071 every 8 connection(s)
I20260811 05:08:47.728528 4482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:47.734144 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.730236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:43063 (local address 127.30.88.254:39073)
0811 05:08:47.730402 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.730407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.731409 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 05:08:47.732653 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.732656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.732692 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.732764 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.733765 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.733768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.733770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.733781 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.733800 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.733808 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.733880 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.734004 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.734040 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:08:47.735021 4636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.730602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39073 (local address 127.30.88.194:43063)
0811 05:08:47.731272 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.731310 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.732465 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.732912 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.732918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.733001 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.733513 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.733514 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.733550 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.733631 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.733882 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.733889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.733893 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.734017 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.734187 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260811 05:08:47.735113 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4482
I20260811 05:08:47.735234 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:47.735522 31075 external_mini_cluster.cc:1428] Setting key 187238c32b5c72255e2f7194cb501a9d
I20260811 05:08:47.736176 4633 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39073
I20260811 05:08:47.736373 4633 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:47.736732 4633 heartbeater.cc:511] Master 127.30.88.254:39073 requested a full tablet report, sending...
I20260811 05:08:47.737402 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--builtin_ntp_servers=127.30.88.212:41787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 05:08:47.737711 4248 ts_manager.cc:194] Registered new tserver with Master: 3b9b407e022d4c13b0ad8ef2072eadee (127.30.88.194:40071)
W20260811 05:08:47.845769 4637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:47.846060 4637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:47.846143 4637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:47.850270 4637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:47.850345 4637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:47.850436 4637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:47.855046 4637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:47.856045 4637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:47.857383 4637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:47.866667 4644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.866655 4646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:47.866655 4643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:47.867043 4637 server_base.cc:1034] running on GCE node
I20260811 05:08:47.867476 4637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:47.868064 4637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:47.869248 4637 hybrid_clock.cc:648] HybridClock initialized: now 1786424927869228 us; error 31 us; skew 500 ppm
I20260811 05:08:47.871284 4637 webserver.cc:460] Webserver started at http://127.30.88.195:39441/ using document root <none> and password file <none>
I20260811 05:08:47.871877 4637 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:47.871973 4637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:47.872184 4637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:47.874248 4637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "99fc549f928be572f1349c47787350bb"
server_key_iv: "65a6448cf30586e2a5f2ac3817e416d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.874935 4637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "99fc549f928be572f1349c47787350bb"
server_key_iv: "65a6448cf30586e2a5f2ac3817e416d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.878383 4637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:47.880870 4656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.881978 4637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:08:47.882109 4637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73"
format_stamp: "Formatted at 2026-08-11 05:08:47 on dist-test-slave-7c35"
server_key: "99fc549f928be572f1349c47787350bb"
server_key_iv: "65a6448cf30586e2a5f2ac3817e416d0"
server_key_version: "encryptionkey@0"
I20260811 05:08:47.882216 4637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:47.887357 4637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:47.887965 4637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:47.888146 4637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:47.888741 4637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:47.889622 4637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:47.889695 4637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.889760 4637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:47.889807 4637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:47.898578 4637 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:41425
I20260811 05:08:47.898604 4804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:41425 every 8 connection(s)
I20260811 05:08:47.899474 4637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:47.904065 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4637
I20260811 05:08:47.904182 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:47.904484 31075 external_mini_cluster.cc:1428] Setting key b3d67eb5b8a1cf58db1eb66d52597a91
I20260811 05:08:47.905726 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.901238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:55713 (local address 127.30.88.254:39073)
0811 05:08:47.901377 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.901381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.902388 (+ 1007us) server_negotiation.cc:407] Connection header received
0811 05:08:47.903771 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.903773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.903832 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.903946 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.905219 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.905223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.905226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.905240 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.905264 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.905280 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.905373 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.905549 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.905596 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:08:47.906569 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.901615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39073 (local address 127.30.88.195:55713)
0811 05:08:47.902242 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.902281 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.903586 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.904076 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.904082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.904205 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.904863 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.904864 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.904934 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.905018 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.905372 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.905383 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.905399 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.905588 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.905810 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":112,"threads_started":1}
I20260811 05:08:47.907604 4805 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39073
I20260811 05:08:47.907801 4805 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:47.908134 4805 heartbeater.cc:511] Master 127.30.88.254:39073 requested a full tablet report, sending...
I20260811 05:08:47.909025 4248 ts_manager.cc:194] Registered new tserver with Master: f85a7f3fd8ec40a1ba185a66a9a6ae73 (127.30.88.195:41425)
I20260811 05:08:47.918257 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:08:47.921777 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.920278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.30.88.254:39073)
0811 05:08:47.920395 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.920398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.920606 (+ 208us) server_negotiation.cc:407] Connection header received
0811 05:08:47.920699 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.920701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.920733 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.920802 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.921505 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.921507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.921508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.921516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.921531 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.921537 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.921596 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.921637 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.921666 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:08:47.924510 4248 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:47.927534 4248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:47.938740 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.935686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:45789 (local address 127.0.0.1:59682)
0811 05:08:47.936033 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.936086 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.936187 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.936821 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.936827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.936908 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.937477 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.937479 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.937497 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.937567 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.938410 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.938419 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.938424 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.938536 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.938600 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260811 05:08:47.938751 4822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.936041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:40071 (local address 127.0.0.1:43256)
0811 05:08:47.936379 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.936392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.936461 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.937371 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.937375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.937393 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.937827 (+ 434us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.937832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.937845 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.937877 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.938472 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.938478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.938484 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.938563 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.938606 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":112,"threads_started":1}
I20260811 05:08:47.939062 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.935876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.30.88.193:45789)
0811 05:08:47.936330 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.936342 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.936368 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:47.936425 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.936431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.936645 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.937202 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.938212 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.938219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.938232 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.938246 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.938286 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.938312 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.938448 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.938549 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.938616 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":126,"threads_started":1}
I20260811 05:08:47.939441 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.936476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.30.88.194:40071)
0811 05:08:47.936962 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.936974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.937000 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:47.937052 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.937058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.937216 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.937635 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.938272 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.938280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.938292 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.938305 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.938341 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.938365 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.939178 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.939226 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.939291 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":160,"threads_started":1}
I20260811 05:08:47.939719 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.936401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:41425 (local address 127.0.0.1:57024)
0811 05:08:47.936704 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:47.936717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:47.936785 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:47.938119 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:47.938121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:47.938135 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:47.939124 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:47.939128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:47.939140 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:47.939165 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:47.939474 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:47.939478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:47.939481 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:08:47.939563 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:08:47.939595 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
I20260811 05:08:47.940502 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:47.936799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.30.88.195:41425)
0811 05:08:47.937438 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:47.937453 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:47.937479 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:47.937534 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:47.937542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:47.937958 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:47.938109 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:47.939291 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:47.939296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:47.939305 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:47.939316 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:47.939352 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:47.939376 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:47.939481 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:47.940302 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 05:08:47.940365 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260811 05:08:47.941983 4399 tablet_service.cc:1467] Processing CreateTablet for tablet b973865e40eb4169ba05167441f19bf2 (DEFAULT_TABLE table=test-table [id=ae9062346fac4926ac85e8c574349256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.942160 4556 tablet_service.cc:1467] Processing CreateTablet for tablet b973865e40eb4169ba05167441f19bf2 (DEFAULT_TABLE table=test-table [id=ae9062346fac4926ac85e8c574349256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.943048 4399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973865e40eb4169ba05167441f19bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.943207 4556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973865e40eb4169ba05167441f19bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.943029 4717 tablet_service.cc:1467] Processing CreateTablet for tablet b973865e40eb4169ba05167441f19bf2 (DEFAULT_TABLE table=test-table [id=ae9062346fac4926ac85e8c574349256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:47.943943 4717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b973865e40eb4169ba05167441f19bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:47.949158 4826 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Bootstrap starting.
I20260811 05:08:47.950577 4827 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Bootstrap starting.
I20260811 05:08:47.952044 4826 tablet_bootstrap.cc:654] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.952658 4827 tablet_bootstrap.cc:654] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.952942 4826 log.cc:824] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.953455 4827 log.cc:824] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.953634 4828 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Bootstrap starting.
I20260811 05:08:47.955034 4826 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: No bootstrap required, opened a new log
I20260811 05:08:47.955256 4826 ts_tablet_manager.cc:1397] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:08:47.955366 4827 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: No bootstrap required, opened a new log
I20260811 05:08:47.955549 4827 ts_tablet_manager.cc:1397] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:08:47.957046 4828 tablet_bootstrap.cc:654] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:47.957708 4828 log.cc:824] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:47.959414 4828 tablet_bootstrap.cc:492] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: No bootstrap required, opened a new log
I20260811 05:08:47.959610 4828 ts_tablet_manager.cc:1397] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:08:47.959762 4826 raft_consensus.cc:348] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.959762 4827 raft_consensus.cc:348] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.960024 4827 raft_consensus.cc:374] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.960024 4826 raft_consensus.cc:374] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.960078 4827 raft_consensus.cc:729] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9b407e022d4c13b0ad8ef2072eadee, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.960078 4826 raft_consensus.cc:729] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06de5af2b20149269720f5364de2e178, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.960549 4827 consensus_queue.cc:260] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.960549 4826 consensus_queue.cc:260] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.961200 4478 heartbeater.cc:503] Master 127.30.88.254:39073 was elected leader, sending a full tablet report...
I20260811 05:08:47.961481 4826 ts_tablet_manager.cc:1428] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:08:47.961788 4633 heartbeater.cc:503] Master 127.30.88.254:39073 was elected leader, sending a full tablet report...
I20260811 05:08:47.963024 4827 ts_tablet_manager.cc:1428] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 05:08:47.964632 4828 raft_consensus.cc:348] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.964905 4828 raft_consensus.cc:374] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:47.964975 4828 raft_consensus.cc:729] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85a7f3fd8ec40a1ba185a66a9a6ae73, State: Initialized, Role: FOLLOWER
I20260811 05:08:47.965500 4828 consensus_queue.cc:260] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:47.966503 4828 ts_tablet_manager.cc:1428] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 05:08:47.966873 4805 heartbeater.cc:503] Master 127.30.88.254:39073 was elected leader, sending a full tablet report...
W20260811 05:08:47.979377 4635 tablet.cc:2369] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:48.021962 4833 raft_consensus.cc:482] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:48.022143 4833 raft_consensus.cc:504] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:48.023267 4833 leader_election.cc:288] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06de5af2b20149269720f5364de2e178 (127.30.88.193:45789), f85a7f3fd8ec40a1ba185a66a9a6ae73 (127.30.88.195:41425)
I20260811 05:08:48.025197 4636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.023472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:45789 (local address 127.30.88.194:53715)
0811 05:08:48.023598 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.023613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:48.023692 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:48.023967 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:48.023970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:48.024012 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:48.024537 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:48.024539 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:48.024549 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:48.024576 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:48.024879 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:48.024883 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:48.024886 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:08:48.025009 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:08:48.025049 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 05:08:48.025206 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.023580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:53715 (local address 127.30.88.193:45789)
0811 05:08:48.023726 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.023731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.023750 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:08:48.023797 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.023803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.023847 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.023949 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.024700 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.024703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.024706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.024717 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.024743 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.024756 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.024870 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.025030 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.025078 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:48.025681 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.023896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:59583 (local address 127.30.88.195:41425)
0811 05:08:48.024058 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.024063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.024272 (+ 209us) server_negotiation.cc:407] Connection header received
0811 05:08:48.024414 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.024416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.024448 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.024536 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.025209 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.025216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.025220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.025249 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.025270 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.025281 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.025342 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.025533 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.025576 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:08:48.025907 4422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b973865e40eb4169ba05167441f19bf2" candidate_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06de5af2b20149269720f5364de2e178" is_pre_election: true
I20260811 05:08:48.026028 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.023737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:41425 (local address 127.30.88.194:59583)
0811 05:08:48.024176 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.024186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:48.024292 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:48.024554 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:48.024556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:48.024570 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:48.025043 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:08:48.025045 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:48.025054 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:08:48.025078 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:48.025413 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:48.025417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:48.025419 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:08:48.025843 (+ 424us) client_negotiation.cc:240] Negotiation successful
0811 05:08:48.025879 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":132,"threads_started":1}
I20260811 05:08:48.026223 4422 raft_consensus.cc:2454] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b9b407e022d4c13b0ad8ef2072eadee in term 0.
I20260811 05:08:48.026600 4749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b973865e40eb4169ba05167441f19bf2" candidate_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" is_pre_election: true
I20260811 05:08:48.026688 4500 leader_election.cc:302] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06de5af2b20149269720f5364de2e178, 3b9b407e022d4c13b0ad8ef2072eadee; no voters:
I20260811 05:08:48.026880 4749 raft_consensus.cc:2454] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b9b407e022d4c13b0ad8ef2072eadee in term 0.
I20260811 05:08:48.027060 4833 raft_consensus.cc:2781] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:48.027199 4833 raft_consensus.cc:482] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:48.027256 4833 raft_consensus.cc:3037] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.028436 4833 raft_consensus.cc:504] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:48.028781 4833 leader_election.cc:288] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [CANDIDATE]: Term 1 election: Requested vote from peers 06de5af2b20149269720f5364de2e178 (127.30.88.193:45789), f85a7f3fd8ec40a1ba185a66a9a6ae73 (127.30.88.195:41425)
I20260811 05:08:48.029106 4422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b973865e40eb4169ba05167441f19bf2" candidate_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06de5af2b20149269720f5364de2e178"
I20260811 05:08:48.029210 4749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b973865e40eb4169ba05167441f19bf2" candidate_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73"
I20260811 05:08:48.029261 4422 raft_consensus.cc:3037] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.029331 4749 raft_consensus.cc:3037] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.030362 4749 raft_consensus.cc:2454] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b9b407e022d4c13b0ad8ef2072eadee in term 1.
I20260811 05:08:48.030362 4422 raft_consensus.cc:2454] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b9b407e022d4c13b0ad8ef2072eadee in term 1.
I20260811 05:08:48.030642 4500 leader_election.cc:302] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06de5af2b20149269720f5364de2e178, 3b9b407e022d4c13b0ad8ef2072eadee; no voters:
I20260811 05:08:48.030814 4833 raft_consensus.cc:2781] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:48.031109 4833 raft_consensus.cc:686] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 1 LEADER]: Becoming Leader. State: Replica: 3b9b407e022d4c13b0ad8ef2072eadee, State: Running, Role: LEADER
I20260811 05:08:48.031440 4833 consensus_queue.cc:237] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } }
I20260811 05:08:48.034631 4247 catalog_manager.cc:5495] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b9b407e022d4c13b0ad8ef2072eadee (127.30.88.194). New cstate: current_term: 1 leader_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b9b407e022d4c13b0ad8ef2072eadee" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 40071 } health_report { overall_health: HEALTHY } } }
I20260811 05:08:48.050900 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.049310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.30.88.194:40071)
0811 05:08:48.049434 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.049440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.049454 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:48.049665 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.049667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.049720 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.049807 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.050436 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.050439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.050442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.050453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.050470 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.050486 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.050581 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.050688 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.050723 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:48.056224 4422 raft_consensus.cc:1261] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 1 FOLLOWER]: Refusing update from remote peer 3b9b407e022d4c13b0ad8ef2072eadee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:48.056257 4749 raft_consensus.cc:1261] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 1 FOLLOWER]: Refusing update from remote peer 3b9b407e022d4c13b0ad8ef2072eadee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:48.056911 4833 consensus_queue.cc:1035] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [LEADER]: Connected to new peer: Peer: permanent_uuid: "06de5af2b20149269720f5364de2e178" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 45789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:48.057123 4837 consensus_queue.cc:1035] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85a7f3fd8ec40a1ba185a66a9a6ae73" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:08:48.057982 4480 tablet.cc:2369] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:48.072494 4247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:48.072672 4247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:48.075909 4247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cca95982a4a4106bbf64c7a9ef5d3f0: Sending DeleteTablet for 3 replicas of tablet b973865e40eb4169ba05167441f19bf2
I20260811 05:08:48.076683 4399 tablet_service.cc:1514] Processing DeleteTablet for tablet b973865e40eb4169ba05167441f19bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:48 UTC) from {username='slave'} at 127.0.0.1:59682
I20260811 05:08:48.077198 4853 tablet_replica.cc:330] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: stopping tablet replica
I20260811 05:08:48.077473 4853 raft_consensus.cc:2229] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:48.077649 4717 tablet_service.cc:1514] Processing DeleteTablet for tablet b973865e40eb4169ba05167441f19bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:48 UTC) from {username='slave'} at 127.0.0.1:57024
I20260811 05:08:48.077780 4853 raft_consensus.cc:2258] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:48.078209 4855 tablet_replica.cc:330] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: stopping tablet replica
I20260811 05:08:48.078500 4556 tablet_service.cc:1514] Processing DeleteTablet for tablet b973865e40eb4169ba05167441f19bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:48 UTC) from {username='slave'} at 127.0.0.1:43256
I20260811 05:08:48.078567 4855 raft_consensus.cc:2229] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:48.078965 4855 raft_consensus.cc:2258] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:48.079319 4853 ts_tablet_manager.cc:1905] T b973865e40eb4169ba05167441f19bf2 P 06de5af2b20149269720f5364de2e178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:48.079840 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4320
I20260811 05:08:48.082530 4855 ts_tablet_manager.cc:1905] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:48.085459 4856 tablet_replica.cc:330] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: stopping tablet replica
I20260811 05:08:48.085815 4856 raft_consensus.cc:2229] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:48.086208 4856 raft_consensus.cc:2258] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:48.086720 4855 ts_tablet_manager.cc:1918] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:48.087069 4855 log.cc:1196] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/b973865e40eb4169ba05167441f19bf2
I20260811 05:08:48.087530 4855 ts_tablet_manager.cc:1939] T b973865e40eb4169ba05167441f19bf2 P f85a7f3fd8ec40a1ba185a66a9a6ae73: Deleting consensus metadata
I20260811 05:08:48.087855 4856 ts_tablet_manager.cc:1905] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:48.088940 4231 catalog_manager.cc:4841] TS f85a7f3fd8ec40a1ba185a66a9a6ae73 (127.30.88.195:41425): tablet b973865e40eb4169ba05167441f19bf2 (table test-table [id=ae9062346fac4926ac85e8c574349256]) successfully deleted
I20260811 05:08:48.090583 4856 ts_tablet_manager.cc:1918] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:48.090698 4856 log.cc:1196] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/b973865e40eb4169ba05167441f19bf2
I20260811 05:08:48.091091 4856 ts_tablet_manager.cc:1939] T b973865e40eb4169ba05167441f19bf2 P 3b9b407e022d4c13b0ad8ef2072eadee: Deleting consensus metadata
I20260811 05:08:48.092053 4233 catalog_manager.cc:4841] TS 3b9b407e022d4c13b0ad8ef2072eadee (127.30.88.194:40071): tablet b973865e40eb4169ba05167441f19bf2 (table test-table [id=ae9062346fac4926ac85e8c574349256]) successfully deleted
W20260811 05:08:48.092912 4231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:45789 (error 108)
I20260811 05:08:48.093014 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4482
I20260811 05:08:48.094933 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.094524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:45789 (local address 127.0.0.1:59694)
0811 05:08:48.094722 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.094804 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:45789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 05:08:48.095112 4231 catalog_manager.cc:4584] TS 06de5af2b20149269720f5364de2e178 (127.30.88.193:45789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b973865e40eb4169ba05167441f19bf2: Network error: Client connection negotiation failed: client connection to 127.30.88.193:45789: connect: Connection refused (error 111)
I20260811 05:08:48.100728 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4637
I20260811 05:08:48.108726 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4211
2026-08-11T05:08:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (950 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 05:08:48.131246 31075 mini_oidc.cc:152] Starting JWKS server
I20260811 05:08:48.132769 31075 webserver.cc:460] Webserver started at https://127.0.0.1:39275/ using document root <none> and password file <none>
I20260811 05:08:48.132900 31075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:08:48.133113 31075 webserver.cc:460] Webserver started at http://127.0.0.1:37385/ using document root <none> and password file <none>
2026-08-11T05:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:48Z Disabled control of system clock
I20260811 05:08:48.142696 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38775
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39275/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/testchainca.cert with env {}
W20260811 05:08:48.248114 4865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:48.248325 4865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:48.248358 4865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:48.251940 4865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:48.251981 4865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:48.251994 4865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:48.252008 4865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:48.252020 4865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:08:48.252033 4865 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:08:48.252048 4865 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39275/jwks.json
I20260811 05:08:48.256669 4865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39275/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:48.257462 4865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:48.258213 4865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:48.265609 4870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:48.265609 4874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:48.265591 4871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:48.265877 4865 server_base.cc:1034] running on GCE node
I20260811 05:08:48.266609 4865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:48.267727 4865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:48.268915 4865 hybrid_clock.cc:648] HybridClock initialized: now 1786424928268884 us; error 47 us; skew 500 ppm
I20260811 05:08:48.270500 4865 webserver.cc:460] Webserver started at http://127.30.88.254:40583/ using document root <none> and password file <none>
I20260811 05:08:48.271091 4865 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:48.271181 4865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:48.271404 4865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:48.273138 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "c7cff68b761b46ed9035855ae215f841"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "999c8f75fe959aa1843acbb300d8cfe2"
server_key_iv: "f07b3fa0d56f5a0c6a213f5a429f1a94"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.273687 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "c7cff68b761b46ed9035855ae215f841"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "999c8f75fe959aa1843acbb300d8cfe2"
server_key_iv: "f07b3fa0d56f5a0c6a213f5a429f1a94"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.277129 4865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:48.279270 4881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:48.280444 4865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:48.280632 4865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "c7cff68b761b46ed9035855ae215f841"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "999c8f75fe959aa1843acbb300d8cfe2"
server_key_iv: "f07b3fa0d56f5a0c6a213f5a429f1a94"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.280745 4865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:48.309273 4865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:48.309906 4865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:48.310134 4865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:48.317303 4865 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38775
I20260811 05:08:48.317329 4942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38775 every 8 connection(s)
I20260811 05:08:48.318250 4865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:48.321158 4943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:48.326336 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841: Bootstrap starting.
I20260811 05:08:48.328328 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4865
I20260811 05:08:48.328424 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:48.328732 31075 external_mini_cluster.cc:1428] Setting key b3b6a55fd4bfb08bae10e1992af2e5c8
I20260811 05:08:48.328742 4943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:48.329571 4943 log.cc:824] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:48.331516 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841: No bootstrap required, opened a new log
I20260811 05:08:48.335669 4943 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } }
I20260811 05:08:48.335968 4943 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:48.336032 4943 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7cff68b761b46ed9035855ae215f841, State: Initialized, Role: FOLLOWER
I20260811 05:08:48.336467 4943 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } }
I20260811 05:08:48.336618 4943 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:48.336690 4943 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:48.336803 4943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.337687 4943 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } }
I20260811 05:08:48.337855 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.329754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.30.88.254:38775)
0811 05:08:48.330434 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.330449 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.330484 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:08:48.331155 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.331175 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.331503 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.331815 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.332169 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.333155 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.334422 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.334927 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.335065 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.335942 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.335947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.335960 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.336430 (+ 470us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.336484 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.336505 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.336616 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.336789 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.337119 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":165,"threads_started":1}
I20260811 05:08:48.338047 4943 leader_election.cc:302] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7cff68b761b46ed9035855ae215f841; no voters:
I20260811 05:08:48.338354 4943 leader_election.cc:288] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:48.338467 4949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:48.338649 4949 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [term 1 LEADER]: Becoming Leader. State: Replica: c7cff68b761b46ed9035855ae215f841, State: Running, Role: LEADER
I20260811 05:08:48.338999 4949 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } }
I20260811 05:08:48.339644 4943 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:48.340121 4949 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7cff68b761b46ed9035855ae215f841. Latest consensus state: current_term: 1 leader_uuid: "c7cff68b761b46ed9035855ae215f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } } }
I20260811 05:08:48.340241 4949 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.340559 4950 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7cff68b761b46ed9035855ae215f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7cff68b761b46ed9035855ae215f841" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38775 } } }
I20260811 05:08:48.340670 4950 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.341012 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:08:48.341236 4954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:48.344929 4954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:48.350589 4954 catalog_manager.cc:1347] Generated new cluster ID: dfb93c57a1c84eb79f89eb2dadf9dae2
I20260811 05:08:48.350665 4954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:48.356839 4954 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:48.357594 4954 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:48.362811 4954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7cff68b761b46ed9035855ae215f841: Generated new TSK 0
I20260811 05:08:48.363477 4954 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:08:48.393183 4978 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 05:08:48.393329 4978 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:08:48.393850 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.371515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.30.88.254:38775)
0811 05:08:48.371641 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.371646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.371844 (+ 198us) server_negotiation.cc:407] Connection header received
0811 05:08:48.372040 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.372042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.372086 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.372165 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:08:48.372468 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.373264 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.374524 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.374736 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.374804 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.375192 (+ 388us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:08:48.393487 (+ 18295us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:08:48.393721 (+ 234us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:08:48.393909 4947 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 05:08:48.395123 4977 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:48.371371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38775 (local address 127.0.0.1:49614)
0811 05:08:48.371742 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.371757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:48.371895 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:48.372212 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:48.372215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:48.372223 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:08:48.372348 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.372355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.373399 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.373403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.374372 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.374411 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.374840 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.374844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.374971 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.375024 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:08:48.393694 (+ 18670us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:08:48.393866 (+ 172us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.88.254:38775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.88.254:38775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 05:08:48.395884 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4865
2026-08-11T05:08:48Z chronyd exiting
I20260811 05:08:48.435894 31075 test_util.cc:175] -----------------------------------------------
I20260811 05:08:48.435981 31075 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786424902138073-31075-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 05:08:48.437105 31075 mini_oidc.cc:152] Starting JWKS server
I20260811 05:08:48.437973 31075 webserver.cc:460] Webserver started at https://127.0.0.1:41715/ using document root <none> and password file <none>
I20260811 05:08:48.438092 31075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:08:48.438395 31075 webserver.cc:460] Webserver started at http://127.0.0.1:41651/ using document root <none> and password file <none>
2026-08-11T05:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:48Z Disabled control of system clock
I20260811 05:08:48.448312 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37851
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:37851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41715/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/testchainca.cert with env {}
W20260811 05:08:48.554405 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:48.554628 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:48.554666 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:48.558319 4984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:48.558362 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:48.558375 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:48.558389 4984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:48.558403 4984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:08:48.558416 4984 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:08:48.558429 4984 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41715/jwks.json
I20260811 05:08:48.563331 4984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:37851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41715/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:48.564172 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:48.565021 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:48.572939 4995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:48.572994 4984 server_base.cc:1034] running on GCE node
W20260811 05:08:48.573010 4993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:48.573192 4990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:48.573757 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:48.574795 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:48.575996 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786424928575969 us; error 39 us; skew 500 ppm
I20260811 05:08:48.577781 4984 webserver.cc:460] Webserver started at http://127.30.88.254:36733/ using document root <none> and password file <none>
I20260811 05:08:48.578299 4984 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:48.578393 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:48.578614 4984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:48.580524 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "4d88985933da4f889675b47eed21b726"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "625e5a8a190cae87555ab18eddf3611a"
server_key_iv: "636c430b6d17f1453dae3bd37684a663"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.581032 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "4d88985933da4f889675b47eed21b726"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "625e5a8a190cae87555ab18eddf3611a"
server_key_iv: "636c430b6d17f1453dae3bd37684a663"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.584478 4984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:48.586571 5001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:48.587672 4984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:48.587769 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "4d88985933da4f889675b47eed21b726"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "625e5a8a190cae87555ab18eddf3611a"
server_key_iv: "636c430b6d17f1453dae3bd37684a663"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.587900 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:48.614414 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:48.614984 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:48.615124 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:48.621152 4984 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:37851
I20260811 05:08:48.621183 5066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:37851 every 8 connection(s)
I20260811 05:08:48.621979 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:48.623596 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 4984
I20260811 05:08:48.623692 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:48.623986 31075 external_mini_cluster.cc:1428] Setting key 487470a0332684ad7f709ba4f7d94b30
I20260811 05:08:48.625428 5067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:48.631441 5067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726: Bootstrap starting.
I20260811 05:08:48.632725 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.625267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.30.88.254:37851)
0811 05:08:48.625866 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.625882 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.625915 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:08:48.626537 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.626551 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.626982 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.627253 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.627577 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.628510 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.629603 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.630019 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.630154 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.631096 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.631100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.631111 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.631478 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.631531 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.631576 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.631665 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.631815 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.632097 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":140,"threads_started":1}
I20260811 05:08:48.633726 5067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:48.634552 5067 log.cc:824] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:48.635450 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:08:48.636514 5067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726: No bootstrap required, opened a new log
I20260811 05:08:48.639736 5067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } }
I20260811 05:08:48.639950 5067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:48.640027 5067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d88985933da4f889675b47eed21b726, State: Initialized, Role: FOLLOWER
I20260811 05:08:48.640516 5067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } }
I20260811 05:08:48.640683 5067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:48.640744 5067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:48.640832 5067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.641575 5067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } }
I20260811 05:08:48.641885 5067 leader_election.cc:302] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d88985933da4f889675b47eed21b726; no voters:
I20260811 05:08:48.642155 5067 leader_election.cc:288] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:48.642267 5073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:48.642505 5073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [term 1 LEADER]: Becoming Leader. State: Replica: 4d88985933da4f889675b47eed21b726, State: Running, Role: LEADER
I20260811 05:08:48.642787 5073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } }
I20260811 05:08:48.643323 5067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:48.644560 5075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d88985933da4f889675b47eed21b726. Latest consensus state: current_term: 1 leader_uuid: "4d88985933da4f889675b47eed21b726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } } }
I20260811 05:08:48.644601 5074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d88985933da4f889675b47eed21b726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d88985933da4f889675b47eed21b726" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37851 } } }
I20260811 05:08:48.644695 5074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.644695 5075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.645143 5082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:48.648039 5082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:48.653110 5082 catalog_manager.cc:1347] Generated new cluster ID: 9752f80d656d4e059563c28db5082950
I20260811 05:08:48.653165 5082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:48.661617 5082 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:48.662340 5082 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:48.671439 5082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d88985933da4f889675b47eed21b726: Generated new TSK 0
I20260811 05:08:48.672041 5082 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:08:48.704428 5099 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 05:08:48.704519 5099 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:08:48.705179 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.680953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.30.88.254:37851)
0811 05:08:48.681084 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.681088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.681264 (+ 176us) server_negotiation.cc:407] Connection header received
0811 05:08:48.681450 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.681452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.681502 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.681609 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:08:48.681848 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.682359 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.683273 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.683486 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.683596 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.683895 (+ 299us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:08:48.704740 (+ 20845us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:08:48.705008 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 05:08:48.705258 5071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 05:08:48.705276 5098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:48.680884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37851 (local address 127.0.0.1:42836)
0811 05:08:48.681180 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.681192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:48.681312 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:48.681610 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:48.681613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:48.681622 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:08:48.681735 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.681748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.682469 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.682474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.683158 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.683166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.683618 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.683622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.683712 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.683733 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:08:48.704918 (+ 21185us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:08:48.705100 (+ 182us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.88.254:37851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260811 05:08:48.705823 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 4984
2026-08-11T05:08:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (310 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 05:08:48.747481 31075 mini_oidc.cc:152] Starting JWKS server
I20260811 05:08:48.748401 31075 webserver.cc:460] Webserver started at https://127.0.0.1:45117/ using document root <none> and password file <none>
I20260811 05:08:48.748512 31075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:08:48.748805 31075 webserver.cc:460] Webserver started at http://127.0.0.1:38581/ using document root <none> and password file <none>
2026-08-11T05:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:48Z Disabled control of system clock
I20260811 05:08:48.758275 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:44495
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:44495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45117/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:08:48.862074 5107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:48.862341 5107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:48.862407 5107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:48.866405 5107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:48.866490 5107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:48.866519 5107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:48.866559 5107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:48.866592 5107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:08:48.866627 5107 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:08:48.866657 5107 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45117/jwks.json
I20260811 05:08:48.871238 5107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:44495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:44495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45117/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:48.872229 5107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:48.873507 5107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:48.882213 5117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:48.882238 5115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:48.882232 5120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:48.882323 5107 server_base.cc:1034] running on GCE node
I20260811 05:08:48.883213 5107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:48.884243 5107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:48.885426 5107 hybrid_clock.cc:648] HybridClock initialized: now 1786424928885404 us; error 46 us; skew 500 ppm
I20260811 05:08:48.887167 5107 webserver.cc:460] Webserver started at http://127.30.88.254:45485/ using document root <none> and password file <none>
I20260811 05:08:48.887686 5107 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:48.887774 5107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:48.887998 5107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:48.890033 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "8a0dca96c8444381b42d80ca012001a3"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "007afd9f6c97975560f4e4901af44265"
server_key_iv: "7c2f279bc2dc39527baeb45f15064530"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.890620 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "8a0dca96c8444381b42d80ca012001a3"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "007afd9f6c97975560f4e4901af44265"
server_key_iv: "7c2f279bc2dc39527baeb45f15064530"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.894200 5107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:48.896543 5126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:48.897578 5107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:08:48.897720 5107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "8a0dca96c8444381b42d80ca012001a3"
format_stamp: "Formatted at 2026-08-11 05:08:48 on dist-test-slave-7c35"
server_key: "007afd9f6c97975560f4e4901af44265"
server_key_iv: "7c2f279bc2dc39527baeb45f15064530"
server_key_version: "encryptionkey@0"
I20260811 05:08:48.897818 5107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:48.910398 5107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:48.911021 5107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:48.911206 5107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:48.917685 5107 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:44495
I20260811 05:08:48.917690 5186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:44495 every 8 connection(s)
I20260811 05:08:48.918494 5107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:48.921361 5187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:48.923718 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5107
I20260811 05:08:48.923862 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:48.924155 31075 external_mini_cluster.cc:1428] Setting key 2a50d7b546bdbd7f4adeceba30de684f
I20260811 05:08:48.927278 5187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3: Bootstrap starting.
I20260811 05:08:48.929764 5187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:48.930517 5187 log.cc:824] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:48.932547 5187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3: No bootstrap required, opened a new log
I20260811 05:08:48.933959 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.925479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.30.88.254:44495)
0811 05:08:48.926134 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.926152 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.926193 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:08:48.926959 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.926982 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.927444 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.927771 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:48.928192 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.929572 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.930450 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.931123 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.931298 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.932100 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:48.932106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:48.932120 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:48.932564 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:48.932635 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:48.932661 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:48.932749 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:48.932897 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 05:08:48.933148 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":91,"threads_started":1}
I20260811 05:08:48.936707 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:08:48.936691 5187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } }
I20260811 05:08:48.936923 5187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:48.937001 5187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a0dca96c8444381b42d80ca012001a3, State: Initialized, Role: FOLLOWER
I20260811 05:08:48.937410 5187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } }
I20260811 05:08:48.937556 5187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:48.937616 5187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:48.937707 5187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:48.938457 5187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } }
I20260811 05:08:48.938766 5187 leader_election.cc:302] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a0dca96c8444381b42d80ca012001a3; no voters:
I20260811 05:08:48.939055 5187 leader_election.cc:288] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:48.939162 5193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:48.939373 5193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [term 1 LEADER]: Becoming Leader. State: Replica: 8a0dca96c8444381b42d80ca012001a3, State: Running, Role: LEADER
I20260811 05:08:48.939654 5193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } }
I20260811 05:08:48.940109 5187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:48.941256 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a0dca96c8444381b42d80ca012001a3. Latest consensus state: current_term: 1 leader_uuid: "8a0dca96c8444381b42d80ca012001a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } } }
I20260811 05:08:48.941262 5195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a0dca96c8444381b42d80ca012001a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0dca96c8444381b42d80ca012001a3" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 44495 } } }
I20260811 05:08:48.941396 5195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.941396 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:48.942052 5205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:48.944567 5205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:48.949697 5205 catalog_manager.cc:1347] Generated new cluster ID: a6212a5209c241deb9b4e14e4f2fcd0d
I20260811 05:08:48.949757 5205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:48.966615 5205 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:48.967555 5205 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:48.979282 5205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a0dca96c8444381b42d80ca012001a3: Generated new TSK 0
I20260811 05:08:48.980067 5205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:08:49.001642 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:48.997014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.30.88.254:44495)
0811 05:08:48.997153 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:48.997160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:48.997359 (+ 199us) server_negotiation.cc:407] Connection header received
0811 05:08:48.997551 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:48.997553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:48.997597 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:48.997699 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:08:48.997943 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.998467 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.999396 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.999618 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.999721 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:49.000023 (+ 302us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:08:49.001330 (+ 1307us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45117/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:08:49.001531 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45117/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 05:08:49.001701 5191 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45117/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 05:08:49.003278 5221 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:48.996940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:44495 (local address 127.0.0.1:36618)
0811 05:08:48.997278 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:48.997289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:48.997407 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:48.997714 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:48.997717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:48.997725 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:08:48.997838 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.997844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.998602 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.998605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.999272 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:48.999280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:48.999743 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:48.999747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:48.999837 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:48.999858 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:08:49.001482 (+ 1624us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45117/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:08:49.003123 (+ 1641us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.88.254:44495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45117/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1459,"thread_start_us":100,"threads_started":1}
I20260811 05:08:49.004109 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5107
E20260811 05:08:49.014581 5222 webserver.cc:533] Webserver: sslize failed:
E20260811 05:08:49.014638 5222 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T05:08:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T05:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:49Z Disabled control of system clock
I20260811 05:08:49.056553 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38521
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38521
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/testchainca.cert with env {}
W20260811 05:08:49.160117 5230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:49.160316 5230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:49.160348 5230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:49.163890 5230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:49.163934 5230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:49.163949 5230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:49.163964 5230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:49.164016 5230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:08:49.164047 5230 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:08:49.164088 5230 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 05:08:49.169348 5230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:49.170198 5230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:49.171144 5230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:08:49.179263 5230 server_base.cc:1034] running on GCE node
W20260811 05:08:49.179282 5241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:49.179196 5238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:49.179196 5237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:49.179896 5230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:49.180846 5230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:49.182013 5230 hybrid_clock.cc:648] HybridClock initialized: now 1786424929181986 us; error 40 us; skew 500 ppm
I20260811 05:08:49.183816 5230 webserver.cc:460] Webserver started at http://127.30.88.254:46245/ using document root <none> and password file <none>
I20260811 05:08:49.184367 5230 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:49.184428 5230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:49.184659 5230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:49.186571 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "1b12ae293711457fb439b409f58b6a5a"
format_stamp: "Formatted at 2026-08-11 05:08:49 on dist-test-slave-7c35"
server_key: "7c07c0c6eccb6c0297aa0e52f89be09f"
server_key_iv: "a9e621bbbb2ec8ab36db848011131375"
server_key_version: "encryptionkey@0"
I20260811 05:08:49.187160 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "1b12ae293711457fb439b409f58b6a5a"
format_stamp: "Formatted at 2026-08-11 05:08:49 on dist-test-slave-7c35"
server_key: "7c07c0c6eccb6c0297aa0e52f89be09f"
server_key_iv: "a9e621bbbb2ec8ab36db848011131375"
server_key_version: "encryptionkey@0"
I20260811 05:08:49.190603 5230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:08:49.193065 5248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:49.194200 5230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:08:49.194346 5230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1b12ae293711457fb439b409f58b6a5a"
format_stamp: "Formatted at 2026-08-11 05:08:49 on dist-test-slave-7c35"
server_key: "7c07c0c6eccb6c0297aa0e52f89be09f"
server_key_iv: "a9e621bbbb2ec8ab36db848011131375"
server_key_version: "encryptionkey@0"
I20260811 05:08:49.194455 5230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:49.207616 5230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:49.208189 5230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:49.208367 5230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:49.215143 5230 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38521
I20260811 05:08:49.215166 5309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38521 every 8 connection(s)
I20260811 05:08:49.215971 5230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:49.218446 5310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:49.221836 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5230
I20260811 05:08:49.221935 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:49.222187 31075 external_mini_cluster.cc:1428] Setting key 562deaecc6e14628bd802478d2b1cab5
I20260811 05:08:49.223603 5310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a: Bootstrap starting.
I20260811 05:08:49.225652 5310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:49.226336 5310 log.cc:824] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:49.228286 5310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a: No bootstrap required, opened a new log
I20260811 05:08:49.231109 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:49.223446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.30.88.254:38521)
0811 05:08:49.224028 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:49.224042 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:49.224075 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:08:49.224692 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:49.224707 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:49.225062 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:49.225390 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:49.225813 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.226811 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.227954 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.228400 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.228502 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:49.229286 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:49.229291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:49.229301 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:08:49.229673 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:49.229800 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:08:49.229846 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:49.229943 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:49.230129 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 05:08:49.230395 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":109,"threads_started":1}
I20260811 05:08:49.232153 5310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } }
I20260811 05:08:49.232393 5310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:49.232461 5310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b12ae293711457fb439b409f58b6a5a, State: Initialized, Role: FOLLOWER
I20260811 05:08:49.232856 5310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } }
I20260811 05:08:49.233021 5310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:49.233088 5310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:49.233174 5310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:49.233763 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:08:49.234174 5310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } }
I20260811 05:08:49.234539 5310 leader_election.cc:302] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b12ae293711457fb439b409f58b6a5a; no voters:
I20260811 05:08:49.234894 5310 leader_election.cc:288] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:49.234997 5315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:49.235173 5315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [term 1 LEADER]: Becoming Leader. State: Replica: 1b12ae293711457fb439b409f58b6a5a, State: Running, Role: LEADER
I20260811 05:08:49.235502 5315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } }
I20260811 05:08:49.236037 5310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:49.237185 5317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b12ae293711457fb439b409f58b6a5a. Latest consensus state: current_term: 1 leader_uuid: "1b12ae293711457fb439b409f58b6a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } } }
I20260811 05:08:49.237296 5317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:49.237556 5316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b12ae293711457fb439b409f58b6a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b12ae293711457fb439b409f58b6a5a" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38521 } } }
I20260811 05:08:49.237630 5316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:49.237697 5327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:49.240378 5327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:49.245230 5327 catalog_manager.cc:1347] Generated new cluster ID: b2cd64e482f34a869ba2e150a6489d9c
I20260811 05:08:49.245294 5327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:49.255275 5327 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:49.256079 5327 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:49.260095 5327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b12ae293711457fb439b409f58b6a5a: Generated new TSK 0
I20260811 05:08:49.260648 5327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:08:50.089537 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:49.267520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.30.88.254:38521)
0811 05:08:49.267648 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:49.267652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:49.267802 (+ 150us) server_negotiation.cc:407] Connection header received
0811 05:08:49.267922 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:49.267924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:49.267973 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:49.268070 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:08:49.268258 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.269024 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.269827 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.270194 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.270307 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:49.270566 (+ 259us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:08:50.089023 (+818457us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:08:50.089363 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47324: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 05:08:50.089607 5344 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:49.267416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38521 (local address 127.0.0.1:47324)
0811 05:08:49.267727 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:49.267738 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:49.267822 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:49.268077 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:49.268079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:49.268085 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:08:49.268166 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:49.268170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.269160 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.269162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:49.269730 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:49.269736 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:49.270326 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:49.270328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:49.270388 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:49.270409 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:08:50.089257 (+818848us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:08:50.089444 (+ 187us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.88.254:38521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
W20260811 05:08:50.089646 5313 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47324: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260811 05:08:50.090312 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5230
2026-08-11T05:08:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1064 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (117 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:50 dist-test-slave-7c35 krb5kdc[5359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:50 dist-test-slave-7c35 krb5kdc[5359](info): set up 2 sockets
Aug 11 05:08:50 dist-test-slave-7c35 krb5kdc[5359](info): commencing operation
krb5kdc: starting...
W20260811 05:08:51.411876 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:51 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:08:51.533239 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41523
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:41523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:51.647130 5377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:51.647416 5377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:51.647488 5377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:51.651538 5377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:51.651620 5377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:51.651655 5377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:51.651693 5377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:51.651726 5377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:51.656891 5377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:41523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:51.657930 5377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:51.659039 5377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:51.667306 5385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:51.667265 5382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:51.667253 5377 server_base.cc:1034] running on GCE node
W20260811 05:08:51.667284 5383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:51.668015 5377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:51.669056 5377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:51.670267 5377 hybrid_clock.cc:648] HybridClock initialized: now 1786424931670246 us; error 30 us; skew 500 ppm
Aug 11 05:08:51 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:51.672997 5377 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:08:51.673878 5377 webserver.cc:460] Webserver started at http://127.30.88.254:34889/ using document root <none> and password file <none>
I20260811 05:08:51.674434 5377 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:51.674530 5377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:51.674762 5377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:51.676784 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "1ee7c9be613b4d92840fad5c1d6dc54d"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "728053bcb23899eea8de8e54ed97f465"
server_key_iv: "751505b49fc09219016195df117c2d69"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.677369 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "1ee7c9be613b4d92840fad5c1d6dc54d"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "728053bcb23899eea8de8e54ed97f465"
server_key_iv: "751505b49fc09219016195df117c2d69"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.680979 5377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:51.683329 5395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:51.684443 5377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:51.684582 5377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1ee7c9be613b4d92840fad5c1d6dc54d"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "728053bcb23899eea8de8e54ed97f465"
server_key_iv: "751505b49fc09219016195df117c2d69"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.684692 5377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:51.698896 5377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:51.701459 5377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:51.701649 5377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:51.708314 5377 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:41523
I20260811 05:08:51.708328 5455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:41523 every 8 connection(s)
I20260811 05:08:51.709203 5377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:51.711840 5456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:51.717003 5456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d: Bootstrap starting.
I20260811 05:08:51.719182 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5377
I20260811 05:08:51.719317 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:51.719399 5456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:51.719677 31075 external_mini_cluster.cc:1428] Setting key 58aa79969812b3c482f4a47ec7bdde4f
I20260811 05:08:51.720393 5456 log.cc:824] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:51.722621 5456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d: No bootstrap required, opened a new log
I20260811 05:08:51.726938 5456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } }
I20260811 05:08:51.727226 5456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:51.727285 5456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee7c9be613b4d92840fad5c1d6dc54d, State: Initialized, Role: FOLLOWER
I20260811 05:08:51.727759 5456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } }
I20260811 05:08:51.727883 5456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:51.727950 5456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:51.728078 5456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 0 FOLLOWER]: Advancing to term 1
Aug 11 05:08:51 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:51.729195 5456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } }
I20260811 05:08:51.729521 5456 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee7c9be613b4d92840fad5c1d6dc54d; no voters:
I20260811 05:08:51.729851 5456 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:51.729920 5461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:51.730168 5461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [term 1 LEADER]: Becoming Leader. State: Replica: 1ee7c9be613b4d92840fad5c1d6dc54d, State: Running, Role: LEADER
I20260811 05:08:51.730510 5461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } }
I20260811 05:08:51.731146 5456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:51.732201 5462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee7c9be613b4d92840fad5c1d6dc54d. Latest consensus state: current_term: 1 leader_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } } }
I20260811 05:08:51.732393 5462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:51.732843 5470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:51.733805 5463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee7c9be613b4d92840fad5c1d6dc54d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41523 } } }
I20260811 05:08:51.733996 5463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:51.735392 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:51.720880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.30.88.254:41523)
0811 05:08:51.721329 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:51.721348 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:51.721388 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:08:51.722219 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:51.722242 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:51.722628 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:51.723009 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:51.723687 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:51.725196 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:51.725990 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:51.726662 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:51.726840 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:51.729764 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:51.729770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:51.729782 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:51.729808 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:51.731802 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:51.732378 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:51.732387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:51.732394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:51.732533 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:51.732849 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:51.732852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:51.732854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:51.733427 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:51.733649 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:51.734181 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 05:08:51.734464 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":124,"threads_started":1}
I20260811 05:08:51.736709 5470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:51.742385 5470 catalog_manager.cc:1347] Generated new cluster ID: 87e43242a87e4d90a7f857ef2d9d3d57
I20260811 05:08:51.742514 5470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:51.770123 5470 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:51.771528 5470 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:51.782737 5470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee7c9be613b4d92840fad5c1d6dc54d: Generated new TSK 0
I20260811 05:08:51.783631 5470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:08:51.815773 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:51.929935 5483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:51.930188 5483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:51.930230 5483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:51.934599 5483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:51.934657 5483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:51.934713 5483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:51.939792 5483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:51.940728 5483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:51.941743 5483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:51.950995 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:51.951010 5492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:51.951025 5483 server_base.cc:1034] running on GCE node
W20260811 05:08:51.951004 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:51.951709 5483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:51.952294 5483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:51.953482 5483 hybrid_clock.cc:648] HybridClock initialized: now 1786424931953453 us; error 41 us; skew 500 ppm
Aug 11 05:08:51 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:51.956281 5483 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:08:51.957147 5483 webserver.cc:460] Webserver started at http://127.30.88.193:40037/ using document root <none> and password file <none>
I20260811 05:08:51.957700 5483 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:51.957791 5483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:51.958033 5483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:51.960120 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "77ceb454457844fabdee769cd96e39d9"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "5f7e8ee16e34e7908c3a393fd4feaf0c"
server_key_iv: "4d9b2a95a2a5ea4a090042f55d8302ff"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.960701 5483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "77ceb454457844fabdee769cd96e39d9"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "5f7e8ee16e34e7908c3a393fd4feaf0c"
server_key_iv: "4d9b2a95a2a5ea4a090042f55d8302ff"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.964417 5483 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:51.966740 5499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:51.967986 5483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:51.968132 5483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "77ceb454457844fabdee769cd96e39d9"
format_stamp: "Formatted at 2026-08-11 05:08:51 on dist-test-slave-7c35"
server_key: "5f7e8ee16e34e7908c3a393fd4feaf0c"
server_key_iv: "4d9b2a95a2a5ea4a090042f55d8302ff"
server_key_version: "encryptionkey@0"
I20260811 05:08:51.968241 5483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:51.993108 5483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:51.996121 5483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:51.996335 5483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:51.996948 5483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:51.997857 5483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:51.997928 5483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:51.997996 5483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:51.998045 5483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.007153 5483 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:41659
I20260811 05:08:52.007184 5629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:41659 every 8 connection(s)
I20260811 05:08:52.008049 5483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:52.012589 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5483
I20260811 05:08:52.012692 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:52.013005 31075 external_mini_cluster.cc:1428] Setting key 7554a4cb441ecdbaa6101315fed48526
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:52.021921 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.009815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:34789 (local address 127.30.88.254:41523)
0811 05:08:52.009963 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.009967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.010759 (+ 792us) server_negotiation.cc:407] Connection header received
0811 05:08:52.012168 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.012171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.012220 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.012295 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.013648 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.014198 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.015430 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.015671 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.015784 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.018297 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.018301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.018303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.018326 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.019871 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.020390 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.020395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.020396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.020488 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.020838 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.020842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.020844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.021030 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.021141 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.021620 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.021746 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:52.022758 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.010177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41523 (local address 127.30.88.193:34789)
0811 05:08:52.010608 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:52.010650 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:52.011973 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:52.012469 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:52.012476 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:52.012936 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:52.013467 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.013481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.014353 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.014356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.015297 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.015307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.015821 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.015826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.015993 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.016638 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:52.016654 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:52.018111 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:52.019996 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.020002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.020017 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.020274 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.020584 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.020588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.020590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.020731 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.021140 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:52.021147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:52.021386 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 05:08:52.021577 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 05:08:52.021880 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260811 05:08:52.023975 5630 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41523
I20260811 05:08:52.024214 5630 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:52.024765 5630 heartbeater.cc:511] Master 127.30.88.254:41523 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:08:52.026335 5415 ts_manager.cc:194] Registered new tserver with Master: 77ceb454457844fabdee769cd96e39d9 (127.30.88.193:41659)
I20260811 05:08:52.027529 5415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:34789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:08:52.039708 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:52.154947 5635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:52.155258 5635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:52.155362 5635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:52.159376 5635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:52.159432 5635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:52.159548 5635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:52.164660 5635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:52.165709 5635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:52.166723 5635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:52.176430 5643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:52.176568 5635 server_base.cc:1034] running on GCE node
W20260811 05:08:52.176414 5640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:52.176414 5641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:52.177189 5635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:52.177838 5635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:52.179018 5635 hybrid_clock.cc:648] HybridClock initialized: now 1786424932178982 us; error 46 us; skew 500 ppm
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.181654 5635 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:08:52.182502 5635 webserver.cc:460] Webserver started at http://127.30.88.194:35305/ using document root <none> and password file <none>
I20260811 05:08:52.183151 5635 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:52.183245 5635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:52.183521 5635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:52.185556 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "ec3e866670a143efa243e4c3e8095d4d"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "7917b9e644c512bfcc9e98918f03b19a"
server_key_iv: "b7d407bcf1c13a1ef7609bf095c607e0"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.186131 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "ec3e866670a143efa243e4c3e8095d4d"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "7917b9e644c512bfcc9e98918f03b19a"
server_key_iv: "b7d407bcf1c13a1ef7609bf095c607e0"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.190024 5635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:08:52.192484 5650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.193691 5635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:52.193828 5635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "ec3e866670a143efa243e4c3e8095d4d"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "7917b9e644c512bfcc9e98918f03b19a"
server_key_iv: "b7d407bcf1c13a1ef7609bf095c607e0"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.193948 5635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:52.212875 5635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:52.215902 5635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:52.216144 5635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:52.216770 5635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:52.217621 5635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:52.217694 5635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.217768 5635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:52.217811 5635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.227195 5635 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:44515
I20260811 05:08:52.227224 5776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:44515 every 8 connection(s)
I20260811 05:08:52.228082 5635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:52.236632 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5635
I20260811 05:08:52.236759 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:52.237046 31075 external_mini_cluster.cc:1428] Setting key 533d93cc6eef3895e6b4b2bba5299bb0
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:52.242569 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.229778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:58145 (local address 127.30.88.254:41523)
0811 05:08:52.229916 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.229920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.230669 (+ 749us) server_negotiation.cc:407] Connection header received
0811 05:08:52.232314 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.232316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.232381 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.232460 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.233974 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.234495 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.235691 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.235918 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.236029 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.238805 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.238809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.238811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.238838 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.240540 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.241077 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.241082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.241084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.241164 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.241497 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.241501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.241503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.241681 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.241787 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.242265 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.242389 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:08:52.243378 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.230071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41523 (local address 127.30.88.194:58145)
0811 05:08:52.230517 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:52.230557 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:52.232028 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:52.232625 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:52.232631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:52.232971 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:52.233808 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.233825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.234625 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.234628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.235559 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.235568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.236067 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.236072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.236240 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.236908 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:52.236924 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:52.238648 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:52.240665 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.240670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.240680 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.240969 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.241256 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.241260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.241262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.241395 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.241787 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:52.241795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:52.242038 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 05:08:52.242224 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 05:08:52.242528 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":110,"threads_started":1}
I20260811 05:08:52.244493 5777 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41523
I20260811 05:08:52.244727 5777 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:52.245184 5777 heartbeater.cc:511] Master 127.30.88.254:41523 requested a full tablet report, sending...
I20260811 05:08:52.246222 5415 ts_manager.cc:194] Registered new tserver with Master: ec3e866670a143efa243e4c3e8095d4d (127.30.88.194:44515)
I20260811 05:08:52.247079 5415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:58145
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:08:52.265767 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:52.377684 5782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:52.377902 5782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:52.377940 5782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:52.381840 5782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:52.381887 5782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:52.381939 5782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:52.386830 5782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:52.387892 5782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:52.388933 5782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:52.398231 5791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:52.398241 5787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:52.398350 5789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:52.398579 5782 server_base.cc:1034] running on GCE node
I20260811 05:08:52.399075 5782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:52.399729 5782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:52.400883 5782 hybrid_clock.cc:648] HybridClock initialized: now 1786424932400854 us; error 42 us; skew 500 ppm
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.408424 5782 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:08:52.409318 5782 webserver.cc:460] Webserver started at http://127.30.88.195:38379/ using document root <none> and password file <none>
I20260811 05:08:52.409876 5782 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:52.409969 5782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:52.410223 5782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:52.412371 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "0609d9e569664bec8a0f4a3f9b80c062"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "81d9134a1156be3aaedfad95fa4cda58"
server_key_iv: "82357d1fa70de92d86d4802a494dd2af"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.412930 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "0609d9e569664bec8a0f4a3f9b80c062"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "81d9134a1156be3aaedfad95fa4cda58"
server_key_iv: "82357d1fa70de92d86d4802a494dd2af"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.416553 5782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:08:52.418713 5799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.419941 5782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:52.420169 5782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "0609d9e569664bec8a0f4a3f9b80c062"
format_stamp: "Formatted at 2026-08-11 05:08:52 on dist-test-slave-7c35"
server_key: "81d9134a1156be3aaedfad95fa4cda58"
server_key_iv: "82357d1fa70de92d86d4802a494dd2af"
server_key_version: "encryptionkey@0"
I20260811 05:08:52.420274 5782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:52.448385 5782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:52.451453 5782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:52.451668 5782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:52.452330 5782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:52.453169 5782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:52.453244 5782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.453315 5782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:52.453366 5782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:52.461833 5782 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:34797
I20260811 05:08:52.461858 5933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:34797 every 8 connection(s)
I20260811 05:08:52.462749 5782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:08:52.472453 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5782
I20260811 05:08:52.472601 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:52.472900 31075 external_mini_cluster.cc:1428] Setting key abf339603b7c941084f587bfd066f072
I20260811 05:08:52.476078 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.464579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:45035 (local address 127.30.88.254:41523)
0811 05:08:52.464733 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.464737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.465492 (+ 755us) server_negotiation.cc:407] Connection header received
0811 05:08:52.466839 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.466877 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.466950 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.467052 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.468141 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.468642 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.469670 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.469869 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.469964 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.472389 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.472393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.472395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.472425 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.474072 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.474635 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.474638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.474640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.474727 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.475175 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.475178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.475180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.475348 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.475427 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.475868 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.475954 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:08:52.476859 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.464928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41523 (local address 127.30.88.195:45035)
0811 05:08:52.465350 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:52.465389 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:52.466642 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:52.467235 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:52.467242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:52.467569 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:52.468000 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.468013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.468760 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.468763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.469552 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:52.469561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.470027 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.470031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:52.470224 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.470788 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:52.470803 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:52.472233 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:52.474220 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.474225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.474238 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.474534 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.474818 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:52.474869 (+ 51us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:52.474871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:52.475070 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:52.475442 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:52.475446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:52.475681 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 05:08:52.475820 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 05:08:52.476045 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260811 05:08:52.477887 5934 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41523
I20260811 05:08:52.478113 5934 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:52.478566 5934 heartbeater.cc:511] Master 127.30.88.254:41523 requested a full tablet report, sending...
I20260811 05:08:52.479650 5415 ts_manager.cc:194] Registered new tserver with Master: 0609d9e569664bec8a0f4a3f9b80c062 (127.30.88.195:34797)
I20260811 05:08:52.480191 5415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:45035
I20260811 05:08:52.486586 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/test-admin.keytab.
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.504966 31075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 05:08:52.507961 5944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:52.506447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.30.88.193:41659)
0811 05:08:52.506787 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.506801 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.506825 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:08:52.506955 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.506961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.507159 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.507289 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.507778 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43980: BlockingRecv error: recv got EOF from 127.0.0.1:43980 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":150,"threads_started":1}
W20260811 05:08:52.507998 31075 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.509208 31075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.510039 31075 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:52.520294 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.511048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.30.88.193:41659)
0811 05:08:52.511169 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.511172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.511185 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:52.511306 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.511308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.511345 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.511412 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.511967 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.512732 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.513604 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.513953 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.514033 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.516558 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.516568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.516580 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.516605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.518484 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.518998 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.519004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.519005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.519093 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.519409 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.519411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.519412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.519669 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.519819 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.520016 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.520149 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 05:08:52.524485 5944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:52.523315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.30.88.193:41659)
0811 05:08:52.523491 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.523494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.523586 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:08:52.523726 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.523756 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.523801 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.523865 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.524395 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44012: BlockingRecv error: recv got EOF from 127.0.0.1:44012 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 05:08:52.524819 31075 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc)
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.525967 31075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.527002 31075 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:52.535997 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.527896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.30.88.193:41659)
0811 05:08:52.528021 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.528026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.528126 (+ 100us) server_negotiation.cc:407] Connection header received
0811 05:08:52.528287 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.528291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.528333 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.528432 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.528942 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.529492 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.530315 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.530571 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.530721 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.533043 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.533048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.533051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.533083 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.534646 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.535073 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.535076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.535077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.535158 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.535367 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.535369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.535370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.535513 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.535607 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.535781 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.535884 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 05:08:52.538542 5944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:08:52.537762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.30.88.193:41659)
0811 05:08:52.537881 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.537884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.537926 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:08:52.538021 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.538023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.538054 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.538113 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.538434 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44036: BlockingRecv error: recv got EOF from 127.0.0.1:44036 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 05:08:52.538642 31075 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786424902138073-31075-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.539697 31075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:52.540580 31075 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:08:52.549181 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:52.541396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.30.88.193:41659)
0811 05:08:52.541519 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:52.541523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:52.541632 (+ 109us) server_negotiation.cc:407] Connection header received
0811 05:08:52.541781 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:52.541784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:52.541842 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:52.541934 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:52.542460 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.542967 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.543775 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:52.544005 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:52.544124 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:52.546286 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:52.546289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:52.546291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:52.546310 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:52.547583 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.548046 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.548052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.548056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.548140 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:52.548404 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:52.548406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:52.548407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:52.548568 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:52.548661 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:52.548894 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 05:08:52.548977 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:08:52.550160 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5483
I20260811 05:08:52.558612 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5635
I20260811 05:08:52.568075 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5782
I20260811 05:08:52.576159 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5377
2026-08-11T05:08:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2246 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5962](info): set up 2 sockets
Aug 11 05:08:52 dist-test-slave-7c35 krb5kdc[5962](info): commencing operation
krb5kdc: starting...
W20260811 05:08:53.712970 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:53 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab.
Entry for principal oryx/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab.
I20260811 05:08:53.834344 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38967
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab
--principal=oryx/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:53.941269 5978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:53.941576 5978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:53.941668 5978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:53.945412 5978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:53.945504 5978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:53.945544 5978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:53.945603 5978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:53.945642 5978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:53.950454 5978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab
--principal=oryx/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:53.951442 5978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:53.952384 5978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:53.960556 5983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:53.960618 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:53.960546 5987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:53.960820 5978 server_base.cc:1034] running on GCE node
I20260811 05:08:53.961359 5978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:53.962400 5978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:53.963573 5978 hybrid_clock.cc:648] HybridClock initialized: now 1786424933963546 us; error 40 us; skew 500 ppm
Aug 11 05:08:53 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:53.966183 5978 init.cc:377] Logged in from keytab as oryx/127.30.88.254@KRBTEST.COM (short username oryx)
I20260811 05:08:53.967103 5978 webserver.cc:460] Webserver started at http://127.30.88.254:40757/ using document root <none> and password file <none>
I20260811 05:08:53.967633 5978 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:53.967717 5978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:53.967942 5978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:53.969774 5978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "df65814267fc41b58ce8a3189c7b4d57"
format_stamp: "Formatted at 2026-08-11 05:08:53 on dist-test-slave-7c35"
server_key: "02f2f716fc4fce474fd7292d45a0004b"
server_key_iv: "66eaea79735808f3f60a03588970d8a9"
server_key_version: "encryptionkey@0"
I20260811 05:08:53.970366 5978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "df65814267fc41b58ce8a3189c7b4d57"
format_stamp: "Formatted at 2026-08-11 05:08:53 on dist-test-slave-7c35"
server_key: "02f2f716fc4fce474fd7292d45a0004b"
server_key_iv: "66eaea79735808f3f60a03588970d8a9"
server_key_version: "encryptionkey@0"
I20260811 05:08:53.973829 5978 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:53.976101 5997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:53.977135 5978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:53.977265 5978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "df65814267fc41b58ce8a3189c7b4d57"
format_stamp: "Formatted at 2026-08-11 05:08:53 on dist-test-slave-7c35"
server_key: "02f2f716fc4fce474fd7292d45a0004b"
server_key_iv: "66eaea79735808f3f60a03588970d8a9"
server_key_version: "encryptionkey@0"
I20260811 05:08:53.977365 5978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:53.990161 5978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:53.992803 5978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:53.993005 5978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:53.999778 5978 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38967
I20260811 05:08:54.000066 6055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38967 every 8 connection(s)
I20260811 05:08:54.000694 5978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:54.003345 6056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:54.008194 6056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Bootstrap starting.
I20260811 05:08:54.010376 6056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:54.010605 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 5978
I20260811 05:08:54.010754 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:54.011075 31075 external_mini_cluster.cc:1428] Setting key 28d8dd3cd665e46d65fd03076f8a2a61
I20260811 05:08:54.011191 6056 log.cc:824] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:54.013091 6056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: No bootstrap required, opened a new log
I20260811 05:08:54.017031 6056 raft_consensus.cc:348] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } }
I20260811 05:08:54.017277 6056 raft_consensus.cc:374] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:54.017343 6056 raft_consensus.cc:729] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df65814267fc41b58ce8a3189c7b4d57, State: Initialized, Role: FOLLOWER
I20260811 05:08:54.017737 6056 consensus_queue.cc:260] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } }
I20260811 05:08:54.017894 6056 raft_consensus.cc:388] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:54.018005 6056 raft_consensus.cc:482] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:54.018118 6056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:54.019321 6056 raft_consensus.cc:504] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } }
I20260811 05:08:54.019667 6056 leader_election.cc:302] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df65814267fc41b58ce8a3189c7b4d57; no voters:
I20260811 05:08:54.019948 6056 leader_election.cc:288] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:54.020043 6062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:54.020380 6062 raft_consensus.cc:686] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 LEADER]: Becoming Leader. State: Replica: df65814267fc41b58ce8a3189c7b4d57, State: Running, Role: LEADER
I20260811 05:08:54.020772 6062 consensus_queue.cc:237] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } }
I20260811 05:08:54.021180 6056 sys_catalog.cc:564] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:54.022557 6065 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df65814267fc41b58ce8a3189c7b4d57. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } }
I20260811 05:08:54.022687 6065 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:54.023023 6063 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } }
I20260811 05:08:54.023114 6063 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:54.026177 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.012231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.30.88.254:38967)
0811 05:08:54.012688 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.012720 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.012754 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:08:54.013450 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.013468 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.013785 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.014059 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.015048 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.016014 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.017141 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.017751 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.017916 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.020886 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.020893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.020911 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.020941 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.023078 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.023616 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.023625 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.023634 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.023777 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.024109 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.024112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.024116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.024617 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.024882 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.025174 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.025503 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":170,"threads_started":1}
W20260811 05:08:54.026351 6080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:08:54.026469 6080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:08:54.026551 6076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:54.027441 6076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:54.032167 6076 catalog_manager.cc:1347] Generated new cluster ID: 5943aa5979b84068ac3ef5c80bc8ff36
I20260811 05:08:54.032229 6076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:54.045632 6076 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:54.046693 6076 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:54.050804 6076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Generated new TSK 0
I20260811 05:08:54.051436 6076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
I20260811 05:08:54.079195 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:54.185094 6088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:54.185328 6088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:54.185372 6088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:54.189335 6088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:54.189388 6088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:54.189479 6088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:54.194532 6088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:54.195811 6088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:54.196813 6088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:54.206336 6098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.206341 6094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.206411 6093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:54.206795 6088 server_base.cc:1034] running on GCE node
I20260811 05:08:54.207190 6088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:54.207779 6088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:54.208963 6088 hybrid_clock.cc:648] HybridClock initialized: now 1786424934208949 us; error 32 us; skew 500 ppm
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:54.211748 6088 init.cc:377] Logged in from keytab as oryx/127.30.88.193@KRBTEST.COM (short username oryx)
I20260811 05:08:54.212483 6088 webserver.cc:460] Webserver started at http://127.30.88.193:40489/ using document root <none> and password file <none>
I20260811 05:08:54.212968 6088 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:54.213027 6088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:54.213178 6088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:54.215133 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "58f06f5b075943c29216783a19ae558a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "0040ff494b88d45cfa97f7116c5cc180"
server_key_iv: "50bc40d49ad571adf9747e2b40546d04"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.215647 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "58f06f5b075943c29216783a19ae558a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "0040ff494b88d45cfa97f7116c5cc180"
server_key_iv: "50bc40d49ad571adf9747e2b40546d04"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.219090 6088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:08:54.221257 6107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.222440 6088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:54.222532 6088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "58f06f5b075943c29216783a19ae558a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "0040ff494b88d45cfa97f7116c5cc180"
server_key_iv: "50bc40d49ad571adf9747e2b40546d04"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.222608 6088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:54.236812 6088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:54.239786 6088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:54.239925 6088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:54.240439 6088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:54.241197 6088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:54.241242 6088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.241277 6088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:54.241292 6088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.250000 6088 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:46243
I20260811 05:08:54.250031 6236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:46243 every 8 connection(s)
I20260811 05:08:54.250885 6088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:54.254789 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6088
I20260811 05:08:54.254971 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:54.255385 31075 external_mini_cluster.cc:1428] Setting key 2a6ad56361a2fe76d0bddd3b4676ebaa
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:54.265408 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.252551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:47953 (local address 127.30.88.254:38967)
0811 05:08:54.252674 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.252678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.253406 (+ 728us) server_negotiation.cc:407] Connection header received
0811 05:08:54.254830 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.254833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.254948 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.255049 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.256616 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.257369 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.258486 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.258932 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.259024 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.261624 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.261631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.261637 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.261672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.263360 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.263840 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.263844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.263846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.263944 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.264268 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.264271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.264273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.264523 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.264619 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.265083 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.265218 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:54.266040 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.252869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:47953)
0811 05:08:54.253273 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.253311 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.254572 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.255354 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.255365 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.255842 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.256456 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.256472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.257504 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.257507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.258350 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.258360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.259097 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.259104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.259333 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.259890 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.259911 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.261459 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.263490 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.263495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.263504 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.263740 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.264050 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.264052 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.264053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.264174 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.264628 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.264632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.264850 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.264995 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.265226 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:08:54.267251 6237 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
I20260811 05:08:54.267496 6237 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:54.267931 6237 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.194@KRBTEST.COM" created.
I20260811 05:08:54.269551 6016 ts_manager.cc:194] Registered new tserver with Master: 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:54.270637 6016 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:47953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
I20260811 05:08:54.284025 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:54.388540 6242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:54.388805 6242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:54.388875 6242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:54.392783 6242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:54.392861 6242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:54.392947 6242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:54.397851 6242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:54.398833 6242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:54.399820 6242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:54.409690 6251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.409641 6253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.409660 6250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:54.409844 6242 server_base.cc:1034] running on GCE node
I20260811 05:08:54.410310 6242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:54.410953 6242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:54.412113 6242 hybrid_clock.cc:648] HybridClock initialized: now 1786424934412101 us; error 23 us; skew 500 ppm
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:54.414780 6242 init.cc:377] Logged in from keytab as oryx/127.30.88.194@KRBTEST.COM (short username oryx)
I20260811 05:08:54.415597 6242 webserver.cc:460] Webserver started at http://127.30.88.194:40935/ using document root <none> and password file <none>
I20260811 05:08:54.416117 6242 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:54.416201 6242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:54.416412 6242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:54.418241 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "912d16ff05c64cb49f23321f33dc971a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "28ff7c6c80c8f896446b2fc952216d96"
server_key_iv: "e77aa6baf43249729bb4cd1b5bcadd8b"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.418749 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "912d16ff05c64cb49f23321f33dc971a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "28ff7c6c80c8f896446b2fc952216d96"
server_key_iv: "e77aa6baf43249729bb4cd1b5bcadd8b"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.422369 6242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:08:54.424574 6261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.425737 6242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:08:54.425920 6242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "912d16ff05c64cb49f23321f33dc971a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "28ff7c6c80c8f896446b2fc952216d96"
server_key_iv: "e77aa6baf43249729bb4cd1b5bcadd8b"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.426023 6242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:54.445762 6242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:54.449051 6242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:54.449247 6242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:54.449807 6242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:54.450621 6242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:54.450693 6242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.450763 6242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:54.450817 6242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.459666 6242 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:46527
I20260811 05:08:54.459697 6391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:46527 every 8 connection(s)
I20260811 05:08:54.460515 6242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:54.470211 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6242
I20260811 05:08:54.470338 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:54.470733 31075 external_mini_cluster.cc:1428] Setting key 02d55646aae2d2bc6e4105e3780b47bc
I20260811 05:08:54.474103 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.462159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:43101 (local address 127.30.88.254:38967)
0811 05:08:54.462288 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.462292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.463111 (+ 819us) server_negotiation.cc:407] Connection header received
0811 05:08:54.464517 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.464521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.464581 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.464685 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.465797 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.466342 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.467542 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.467769 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.467882 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.470241 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.470245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.470249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.470303 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.471981 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.472592 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.472595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.472596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.472671 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.473054 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.473056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.473057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.473218 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.473290 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.473874 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.473987 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:08:54.475239 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.462500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:43101)
0811 05:08:54.462976 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.463016 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.464287 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.464858 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.464865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.465195 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.465650 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.465664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.466468 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.466472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.467406 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.467417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.467921 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.467925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.468101 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.468592 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.468621 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.470059 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.472092 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.472096 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.472106 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.472476 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.472768 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.472774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.472778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.472958 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.473318 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.473328 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.473641 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.473837 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.474275 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
I20260811 05:08:54.476320 6392 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
I20260811 05:08:54.476557 6392 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:54.477082 6392 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
I20260811 05:08:54.478155 6016 ts_manager.cc:194] Registered new tserver with Master: 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527)
I20260811 05:08:54.478729 6016 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:43101
WARNING: no policy specified for oryx/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
I20260811 05:08:54.499444 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:54.609467 6397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:54.609728 6397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:54.609807 6397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:54.613955 6397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:54.614027 6397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:54.614120 6397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:54.619168 6397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:54.620174 6397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:54.621248 6397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:54.631287 6408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.631374 6405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.631381 6406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:54.631521 6397 server_base.cc:1034] running on GCE node
I20260811 05:08:54.631979 6397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:54.632572 6397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:54.633751 6397 hybrid_clock.cc:648] HybridClock initialized: now 1786424934633727 us; error 47 us; skew 500 ppm
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:54.636565 6397 init.cc:377] Logged in from keytab as oryx/127.30.88.195@KRBTEST.COM (short username oryx)
I20260811 05:08:54.637318 6397 webserver.cc:460] Webserver started at http://127.30.88.195:34389/ using document root <none> and password file <none>
I20260811 05:08:54.637820 6397 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:54.637876 6397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:54.638036 6397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:54.639966 6397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "88bd0b78ec53475980c1f8b9752250a4"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "bdef36f2e24ae540d4e538fc8cf236c4"
server_key_iv: "956700d086a147c390b0c4d56178125e"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.640470 6397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "88bd0b78ec53475980c1f8b9752250a4"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "bdef36f2e24ae540d4e538fc8cf236c4"
server_key_iv: "956700d086a147c390b0c4d56178125e"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.644071 6397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:54.646427 6418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.647603 6397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:54.647694 6397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "88bd0b78ec53475980c1f8b9752250a4"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "bdef36f2e24ae540d4e538fc8cf236c4"
server_key_iv: "956700d086a147c390b0c4d56178125e"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.647773 6397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:54.656414 6397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:54.659612 6397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:54.659756 6397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:54.660249 6397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:54.660995 6397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:54.661041 6397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.661077 6397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:54.661106 6397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.669420 6397 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39027
I20260811 05:08:54.669516 6547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39027 every 8 connection(s)
I20260811 05:08:54.670293 6397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:54.675799 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6397
I20260811 05:08:54.675926 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:08:54.676209 31075 external_mini_cluster.cc:1428] Setting key 97c51cd8c860cf6afecf12d6a6d81cee
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:54.683611 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.672071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:58983 (local address 127.30.88.254:38967)
0811 05:08:54.672206 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.672209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.672958 (+ 749us) server_negotiation.cc:407] Connection header received
0811 05:08:54.674265 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.674268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.674314 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.674387 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.675541 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.676336 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.677682 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.677975 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.678096 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.680415 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.680418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.680419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.680442 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.681920 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.682343 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.682346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.682347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.682417 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.682695 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.682697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.682698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.682910 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.682986 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.683401 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.683498 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:08:54.684316 6551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.672403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:58983)
0811 05:08:54.672815 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.672854 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.674075 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.674554 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.674560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.674952 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.675398 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.675410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.676465 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.676468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.677580 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.677589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.678135 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.678139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.678309 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.678792 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.678807 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.680258 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.682013 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.682017 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.682027 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.682259 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.682494 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.682496 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.682498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.682618 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.682997 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.683002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.683216 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.683357 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.683575 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
I20260811 05:08:54.685297 6548 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
I20260811 05:08:54.685530 6548 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:54.685976 6548 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
I20260811 05:08:54.687008 6016 ts_manager.cc:194] Registered new tserver with Master: 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
I20260811 05:08:54.687539 6016 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:58983
I20260811 05:08:54.690119 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.88.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.253.keytab.
Entry for principal oryx/127.30.88.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.253.keytab.
I20260811 05:08:54.718068 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:40321
--webserver_interface=127.30.88.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.253.keytab
--principal=oryx/127.30.88.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38967,127.30.88.253:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:54.832639 6555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:54.832894 6555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:54.832933 6555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:54.836843 6555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:54.836901 6555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:54.836920 6555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:54.836937 6555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:54.836951 6555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:54.842257 6555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38967,127.30.88.253:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.253.keytab
--principal=oryx/127.30.88.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.88.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
W20260811 05:08:54.842430 6555 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.88.254:38967,127.30.88.253:40321'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 05:08:54.843286 6555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:54.844197 6555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:54.853899 6561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.853906 6565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:54.853910 6562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:54.854018 6555 server_base.cc:1034] running on GCE node
I20260811 05:08:54.854693 6555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:54.855791 6555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:54.856953 6555 hybrid_clock.cc:648] HybridClock initialized: now 1786424934856934 us; error 31 us; skew 500 ppm
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:54.859758 6555 init.cc:377] Logged in from keytab as oryx/127.30.88.253@KRBTEST.COM (short username oryx)
I20260811 05:08:54.860589 6555 webserver.cc:460] Webserver started at http://127.30.88.253:37863/ using document root <none> and password file <none>
I20260811 05:08:54.861081 6555 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:54.861137 6555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:54.861294 6555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:54.863296 6555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/instance:
uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "8bfb5663adfb1a537f5195202acac33e"
server_key_iv: "d62b7c768a3e813a37e60e94a7c11716"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.863796 6555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal/instance:
uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "8bfb5663adfb1a537f5195202acac33e"
server_key_iv: "d62b7c768a3e813a37e60e94a7c11716"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.867292 6555 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:54.869558 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:54.870769 6555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:54.870925 6555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "8bfb5663adfb1a537f5195202acac33e"
server_key_iv: "d62b7c768a3e813a37e60e94a7c11716"
server_key_version: "encryptionkey@0"
I20260811 05:08:54.871064 6555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:54.891024 6555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:54.894644 6555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:54.894941 6555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:54.902233 6555 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.253:40321
I20260811 05:08:54.902566 6633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.253:40321 every 8 connection(s)
I20260811 05:08:54.903383 6555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
I20260811 05:08:54.904165 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6555
I20260811 05:08:54.906819 6634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:54.910589 6634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.253@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:54.919414 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.905844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.30.88.253:40321)
0811 05:08:54.906318 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.906335 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.906371 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:08:54.907139 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.907159 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.907482 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.907835 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.908701 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.909710 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.910574 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.911156 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.911341 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.914521 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.914527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.914537 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.914565 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.916267 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.916729 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.916735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.916747 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.916857 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.917164 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.917168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.917170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.917696 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.917903 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.918233 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.918556 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":158,"threads_started":1}
I20260811 05:08:54.922264 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.912342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58620)
0811 05:08:54.912693 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.912744 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.912874 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.913239 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.913246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.913713 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.913919 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.913928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.914737 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.914740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.915773 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.915784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.916246 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.916249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.916362 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.916863 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.916876 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.918416 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.920367 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.920374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.920378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.920588 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.920916 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.920919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.920921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.921019 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.921593 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.921601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.921712 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.921877 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.922070 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
I20260811 05:08:54.922323 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.912436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.30.88.254:38967)
0811 05:08:54.912612 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.912619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.912843 (+ 224us) server_negotiation.cc:407] Connection header received
0811 05:08:54.912989 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.912992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.913060 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.913163 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.914043 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.914597 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.915903 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.916130 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.916242 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.918545 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.918548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.918550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.918575 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.920261 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.920698 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.920702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.920704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.920790 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.921137 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.921140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.921142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.921465 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.921567 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.922028 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.922145 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 05:08:54.923487 6634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424934, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.253@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
I20260811 05:08:54.931488 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.924035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.30.88.253:40321)
0811 05:08:54.924155 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.924159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.924208 (+ 49us) server_negotiation.cc:407] Connection header received
0811 05:08:54.924337 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.924339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.924374 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.924435 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.924781 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.925692 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.926477 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.926707 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.926785 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.928638 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.928641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.928643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.928662 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.929865 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.930257 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.930259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.930260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.930328 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.930628 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.930634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.930637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.930799 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.930967 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.931212 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.931359 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":49}
I20260811 05:08:54.931497 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.923964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56640)
0811 05:08:54.924149 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.924164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.924251 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.924475 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.924477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.924599 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.924689 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.924694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.925797 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.925800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.926375 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.926384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.926801 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.926803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.926918 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.927290 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.927300 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.928520 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.929962 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.929964 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.929965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.930172 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.930441 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.930443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.930445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.930535 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.930958 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.930961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.931053 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.931168 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.931303 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:08:54.934505 6634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Bootstrap starting.
I20260811 05:08:54.936842 6634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:54.937635 6634 log.cc:824] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:54.939659 6634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: No bootstrap required, opened a new log
I20260811 05:08:54.942687 6634 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } }
I20260811 05:08:54.942998 6634 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:54.943080 6634 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f64f2a856d4f23b66e768cf59f84b9, State: Initialized, Role: FOLLOWER
I20260811 05:08:54.943575 6634 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } }
I20260811 05:08:54.944053 6643 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } } }
I20260811 05:08:54.944161 6643 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:54.944816 6634 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:08:54.949532 6654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:08:54.949617 6654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:08:54.952073 6555 master_runner.cc:428] Detected that this master d3f64f2a856d4f23b66e768cf59f84b9 is joining an existing cluster
I20260811 05:08:54.952122 6555 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.88.253:40321
I20260811 05:08:54.962903 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.954450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.30.88.253:40321)
0811 05:08:54.954548 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.954552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.954566 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:54.954662 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.954665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.954737 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.954826 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.955436 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.956229 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957292 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.957509 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957634 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.959288 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.959291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.959293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.959310 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.960965 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.961624 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.961628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.961630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.961719 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.962042 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.962047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.962049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.962230 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.962336 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.962588 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.962730 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":36}
I20260811 05:08:54.962914 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.954132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56656)
0811 05:08:54.954450 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.954464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.954565 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.954893 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.954896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.955148 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.955310 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.955320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.956351 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.956354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.957189 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.957196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.957738 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.957841 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.958504 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.958521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.959147 (+ 626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.961096 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.961100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.961103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.961319 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.961828 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.961832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.961836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.961945 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.962325 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.962328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.962412 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.962529 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.962678 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"spinlock_wait_cycles":6912,"thread_start_us":108,"threads_started":1}
I20260811 05:08:54.963267 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.954126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.30.88.254:38967)
0811 05:08:54.954278 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.954283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.954532 (+ 249us) server_negotiation.cc:407] Connection header received
0811 05:08:54.954684 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.954689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.954756 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.954833 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.955472 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.956009 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957126 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.957478 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957595 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.959295 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.959299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.959302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.959329 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.961417 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.961917 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.961921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.961922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.962016 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.962314 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.962317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.962319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.962521 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.962590 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.962986 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.963106 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:08:54.963981 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.954057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58632)
0811 05:08:54.954450 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.954464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.954579 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.955034 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.955036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.955220 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.955336 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.955344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.956116 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.956121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.956985 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.957005 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.957604 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.957607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.957722 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.958432 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.958447 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.959147 (+ 700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.961518 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.961521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.961524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.961808 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.962124 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.962127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.962129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.962214 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.962662 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.962665 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.962762 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.963641 (+ 879us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.963785 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":25,"spinlock_wait_cycles":2432,"thread_start_us":130,"threads_started":1}
Aug 11 05:08:54 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.254@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
I20260811 05:08:54.978785 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.968395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56664 (local address 127.30.88.253:40321)
0811 05:08:54.968994 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:54.968997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:54.969010 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:54.969104 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:54.969106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:54.969144 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:54.969207 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:54.969959 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.970480 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.971756 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.971962 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.972077 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.974899 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:54.974903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:54.974906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:54.974931 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:54.976686 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.977264 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.977267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.977269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.977355 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:54.977685 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:54.977688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:54.977690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:54.977926 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:54.978037 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:54.978522 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 05:08:54.978631 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530}
I20260811 05:08:54.978997 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:54.968375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56664)
0811 05:08:54.968793 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:54.968854 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:54.968989 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:54.969265 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:54.969271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:54.969715 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:54.969846 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.969854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.970587 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.970591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.971644 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:54.971656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:54.972083 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:54.972087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:54.972192 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:54.972886 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:54.972901 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:54.974713 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:54.976808 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.976821 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.976830 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.977158 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.977461 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:54.977467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:54.977472 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:54.977579 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:54.978071 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:54.978081 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:54.978269 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 05:08:54.978458 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 05:08:54.978685 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":112,"threads_started":1}
I20260811 05:08:54.980906 6016 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.88.253:40321: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
member_type: NON_VOTER
last_known_addr {
host: "127.30.88.253"
port: 40321
}
attrs {
promote: true
}
}
dest_uuid: "df65814267fc41b58ce8a3189c7b4d57"
cas_config_opid_index: -1
I20260811 05:08:54.981726 6016 consensus_queue.cc:237] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.30.88.252@KRBTEST.COM; defaulting to no policy
I20260811 05:08:54.983675 6666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d3f64f2a856d4f23b66e768cf59f84b9 (127.30.88.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } }
I20260811 05:08:54.984436 6604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.252@KRBTEST.COM" created.
I20260811 05:08:54.986616 6666 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.88.253:40321
I20260811 05:08:54.987150 6555 master.cc:537] Master@127.30.88.253:40321 shutting down...
I20260811 05:08:54.987180 6604 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 05:08:54.987788 6666 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } }
I20260811 05:08:54.987983 6666 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:54.987901 6664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:08:54.990262 5999 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 05:08:54.991712 5999 consensus_peers.cc:487] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 -> Peer d3f64f2a856d4f23b66e768cf59f84b9 (127.30.88.253:40321): Couldn't send request to peer d3f64f2a856d4f23b66e768cf59f84b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:08:54.992967 6555 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:54.993382 6555 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:54.993597 6555 tablet_replica.cc:330] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.252.keytab.
Entry for principal oryx/127.30.88.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.252.keytab.
I20260811 05:08:55.006269 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:44803
--webserver_interface=127.30.88.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.252.keytab
--principal=oryx/127.30.88.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:08:55.009697 6555 master.cc:552] Master@127.30.88.253:40321 shutdown complete.
I20260811 05:08:55.009812 6555 master_runner.cc:305] Clearing existing system tablet
I20260811 05:08:55.010139 6555 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.010254 6555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.011942 6555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:55.012986 6673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.013260 6555 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.013406 6555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:55.013463 6555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "8bfb5663adfb1a537f5195202acac33e"
server_key_iv: "d62b7c768a3e813a37e60e94a7c11716"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.016597 6555 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:55.019732 6555 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:08:55.019902 6555 log.cc:1196] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 05:08:55.020422 6555 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Deleting consensus metadata
I20260811 05:08:55.020682 6555 master_runner.cc:315] Copying system tablet from 127.30.88.254:38967
I20260811 05:08:55.022197 6555 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: tablet copy: Beginning tablet copy session from remote peer at address 127.30.88.254:38967
I20260811 05:08:55.035028 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.023023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58636)
0811 05:08:55.025693 (+ 2670us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.025709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.025856 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.026184 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.026187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.026426 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.026770 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.026785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.028008 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.028012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.028945 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.028957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.029684 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.029687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.029791 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.030453 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.030470 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.030948 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.033239 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.033242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.033244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.033498 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.033833 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.033836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.033837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.033930 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.034398 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.034401 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.034502 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.034598 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.034764 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2582,"thread_start_us":85,"threads_started":1}
I20260811 05:08:55.035073 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.023096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.30.88.254:38967)
0811 05:08:55.023232 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.023238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.025826 (+ 2588us) server_negotiation.cc:407] Connection header received
0811 05:08:55.025984 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.025988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.026053 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.026150 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.026987 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.027815 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.029086 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.029466 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.029605 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.031108 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.031112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.031116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.031147 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.033094 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.033622 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.033626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.033628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.033731 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.034046 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.034050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.034051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.034292 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.034403 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.034700 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.034902 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:08:55.036134 6037 tablet_copy_service.cc:140] P df65814267fc41b58ce8a3189c7b4d57: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d3f64f2a856d4f23b66e768cf59f84b9 ({username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:58636)
I20260811 05:08:55.036309 6037 tablet_copy_service.cc:161] P df65814267fc41b58ce8a3189c7b4d57: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d3f64f2a856d4f23b66e768cf59f84b9 at {username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:58636: session id = d3f64f2a856d4f23b66e768cf59f84b9-00000000000000000000000000000000
I20260811 05:08:55.038064 6037 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:08:55.039878 6555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:08:55.042898 6555 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: tablet copy: Starting download of 0 data blocks...
I20260811 05:08:55.043114 6555 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: tablet copy: Starting download of 1 WAL segments...
I20260811 05:08:55.045063 6555 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:08:55.047093 6037 tablet_copy_service.cc:342] P df65814267fc41b58ce8a3189c7b4d57: Request end of tablet copy session d3f64f2a856d4f23b66e768cf59f84b9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:58636
I20260811 05:08:55.047224 6037 tablet_copy_service.cc:433] P df65814267fc41b58ce8a3189c7b4d57: ending tablet copy session d3f64f2a856d4f23b66e768cf59f84b9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d3f64f2a856d4f23b66e768cf59f84b9
I20260811 05:08:55.055235 6555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:55.057073 6680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.059736 6681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.059976 6683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.063742 6555 server_base.cc:1034] running on GCE node
I20260811 05:08:55.064082 6555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.064517 6555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.065688 6555 hybrid_clock.cc:648] HybridClock initialized: now 1786424935065670 us; error 36 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.067673 6555 init.cc:377] Logged in from keytab as oryx/127.30.88.253@KRBTEST.COM (short username oryx)
I20260811 05:08:55.068324 6555 webserver.cc:460] Webserver started at http://127.30.88.253:32965/ using document root <none> and password file <none>
I20260811 05:08:55.068593 6555 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.068708 6555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.070389 6555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:55.071724 6691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.072054 6555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:55.072180 6555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "d3f64f2a856d4f23b66e768cf59f84b9"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "8bfb5663adfb1a537f5195202acac33e"
server_key_iv: "d62b7c768a3e813a37e60e94a7c11716"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.072340 6555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.092240 6555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.094751 6555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.099944 6555 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.253:40321
I20260811 05:08:55.100005 6757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.253:40321 every 8 connection(s)
I20260811 05:08:55.100795 6555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
I20260811 05:08:55.102901 6758 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:08:55.104022 6758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Bootstrap starting.
I20260811 05:08:55.114161 6758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:08:55.114455 6758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Bootstrap complete.
I20260811 05:08:55.115056 6758 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } }
I20260811 05:08:55.115190 6758 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d3f64f2a856d4f23b66e768cf59f84b9, State: Initialized, Role: LEARNER
I20260811 05:08:55.115327 6758 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } }
I20260811 05:08:55.115655 6761 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } }
I20260811 05:08:55.115756 6758 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:55.115871 6761 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:08:55.120151 6774 catalog_manager.cc:1259] Loaded cluster ID: 5943aa5979b84068ac3ef5c80bc8ff36
I20260811 05:08:55.120215 6774 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: loading cluster ID for follower catalog manager: success
I20260811 05:08:55.122462 6774 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: acquiring CA information for follower catalog manager: success
I20260811 05:08:55.124042 6774 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 05:08:55.150069 6671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:55.150334 6671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:55.150372 6671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:55.154618 6671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:55.154672 6671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:55.154691 6671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:55.154712 6671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:55.154727 6671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:55.159909 6671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.252.keytab
--principal=oryx/127.30.88.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:44803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.88.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:55.160840 6671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:55.161801 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:55.170001 6786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.169966 6784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.170010 6780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.170214 6671 server_base.cc:1034] running on GCE node
I20260811 05:08:55.170796 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.171900 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.173090 6671 hybrid_clock.cc:648] HybridClock initialized: now 1786424935173057 us; error 45 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.175879 6671 init.cc:377] Logged in from keytab as oryx/127.30.88.252@KRBTEST.COM (short username oryx)
I20260811 05:08:55.176723 6671 webserver.cc:460] Webserver started at http://127.30.88.252:45119/ using document root <none> and password file <none>
I20260811 05:08:55.177274 6671 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.177364 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.177604 6671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:55.179714 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/instance:
uuid: "6ffa08b485524c069089c5a668e5f427"
format_stamp: "Formatted at 2026-08-11 05:08:55 on dist-test-slave-7c35"
server_key: "68191f77b52d0a2abc5ace1584e497a3"
server_key_iv: "2ddf3f502cde390139e10b0d178855e7"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.180301 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal/instance:
uuid: "6ffa08b485524c069089c5a668e5f427"
format_stamp: "Formatted at 2026-08-11 05:08:55 on dist-test-slave-7c35"
server_key: "68191f77b52d0a2abc5ace1584e497a3"
server_key_iv: "2ddf3f502cde390139e10b0d178855e7"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.183957 6671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:08:55.186197 6793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.187325 6671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:55.187463 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "6ffa08b485524c069089c5a668e5f427"
format_stamp: "Formatted at 2026-08-11 05:08:55 on dist-test-slave-7c35"
server_key: "68191f77b52d0a2abc5ace1584e497a3"
server_key_iv: "2ddf3f502cde390139e10b0d178855e7"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.187568 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.204929 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.207885 6671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:55.208066 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.214211 6671 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.252:44803
I20260811 05:08:55.214481 6851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.252:44803 every 8 connection(s)
I20260811 05:08:55.215194 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
I20260811 05:08:55.217984 6853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:55.221539 6853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:08:55.224653 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6671
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.252@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
I20260811 05:08:55.236642 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.223651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58646 (local address 127.30.88.254:38967)
0811 05:08:55.223782 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.223785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.224506 (+ 721us) server_negotiation.cc:407] Connection header received
0811 05:08:55.225811 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.225816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.225874 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.225990 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.227422 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.228260 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.229628 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.229948 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.230063 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.232638 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.232641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.232644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.232671 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.234278 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.234932 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.234945 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.234946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.235038 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.235403 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.235406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.235407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.235668 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.235757 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.236319 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.236449 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:08:55.237457 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.225630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.30.88.252:44803)
0811 05:08:55.225919 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.225927 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.225988 (+ 61us) server_negotiation.cc:407] Connection header received
0811 05:08:55.226161 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.226167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.226354 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.226530 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.227626 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.228668 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.229804 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.230349 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.230434 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.233095 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.233101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.233109 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.233127 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.234783 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.235288 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.235293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.235295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.235376 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.235654 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.235656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.235657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.236087 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.236283 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.236499 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.236619 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":175,"spinlock_wait_cycles":3456,"thread_start_us":91,"threads_started":1}
I20260811 05:08:55.237457 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.223899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58646)
0811 05:08:55.224300 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.224338 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.225409 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.226169 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.226177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.226616 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.227247 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.227265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.228401 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.228403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.229491 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.229502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.230166 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.230171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.230326 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.230911 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.230937 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.232480 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.234386 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.234395 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.234406 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.234813 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.235131 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.235133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.235135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.235295 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.235769 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.235775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.236088 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.236243 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.236565 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":77,"spinlock_wait_cycles":9472,"thread_start_us":95,"threads_started":1}
I20260811 05:08:55.238734 6853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.252@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
I20260811 05:08:55.246097 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.239151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56678)
0811 05:08:55.239248 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.239261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.239328 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.239744 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.239746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.239862 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.239940 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.239944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.240770 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.240772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.241216 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.241222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.241643 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.241645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.241715 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.242072 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.242081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.243307 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.244939 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.244941 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.244942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.245108 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.245348 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.245350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.245351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.245436 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.245755 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.245757 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.245827 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.245902 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.245994 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 05:08:55.246287 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.239275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.30.88.253:40321)
0811 05:08:55.239527 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.239534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.239548 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:55.239591 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.239594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.239639 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.239710 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.240040 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.240687 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.241333 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.241556 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.241629 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.243433 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.243441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.243443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.243466 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.244848 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.245190 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.245192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.245193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.245266 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.245514 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.245516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.245517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.245678 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.245752 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.245997 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.246117 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260811 05:08:55.247109 6853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.252@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
I20260811 05:08:55.254487 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.247522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60326)
0811 05:08:55.247619 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.247631 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.247697 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.247903 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.247906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.248015 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.248074 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.248078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.248664 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.248666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.249126 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.249132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.249602 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.249607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.249703 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.250133 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.250147 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.251472 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.253264 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.253266 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.253267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.253440 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.253700 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.253702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.253703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.253768 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.254158 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.254160 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.254227 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.254302 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.254384 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 05:08:55.254621 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.247654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.30.88.252:44803)
0811 05:08:55.247752 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.247755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.247765 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:08:55.247790 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.247792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.247820 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.247877 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.248157 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.248584 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.249214 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.249469 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.249582 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.251606 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.251609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.251612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.251633 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.253157 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.253529 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.253532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.253534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.253613 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.253851 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.253854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.253856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.254051 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.254165 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.254387 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.254499 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:55.257649 6853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Bootstrap starting.
I20260811 05:08:55.259886 6853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:55.260589 6853 log.cc:824] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:55.262338 6853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: No bootstrap required, opened a new log
I20260811 05:08:55.265340 6853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } }
I20260811 05:08:55.265532 6853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:55.265609 6853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffa08b485524c069089c5a668e5f427, State: Initialized, Role: FOLLOWER
I20260811 05:08:55.266017 6853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } }
I20260811 05:08:55.266459 6859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } } }
I20260811 05:08:55.266592 6859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:55.267217 6853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:08:55.271683 6872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:08:55.271766 6872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:08:55.272836 6237 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
I20260811 05:08:55.275252 6671 master_runner.cc:428] Detected that this master 6ffa08b485524c069089c5a668e5f427 is joining an existing cluster
I20260811 05:08:55.275314 6671 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.88.252:44803
I20260811 05:08:55.286669 6878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.277614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58658)
0811 05:08:55.277883 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.277897 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.278029 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.278381 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.278383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.278628 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.278757 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.278765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.279924 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.279928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.280842 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.280852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.281606 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281637 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.281749 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.282302 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.282317 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.282787 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.285057 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.285063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.285264 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.285580 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.285585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.285677 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.286102 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.286105 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.286198 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.286343 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.286478 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260811 05:08:55.286705 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.277659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.30.88.254:38967)
0811 05:08:55.277776 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.277782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.277963 (+ 181us) server_negotiation.cc:407] Connection header received
0811 05:08:55.278139 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.278143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.278192 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.278241 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.278935 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.279725 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281038 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.281362 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281451 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.282973 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.282977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.282980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.283006 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.284940 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285358 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.285362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.285364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.285463 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285782 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.285786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.285788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.285981 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.286086 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.286406 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.286538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:08:55.287082 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.278563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.30.88.252:44803)
0811 05:08:55.278660 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.278664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.278913 (+ 249us) server_negotiation.cc:407] Connection header received
0811 05:08:55.279042 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.279044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.279074 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.279117 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.279638 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.280371 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281432 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.282353 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.282475 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.283508 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.283513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.283518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.283540 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.284952 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285603 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.285606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.285608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.285684 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.286296 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.286302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.286304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.286480 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.286568 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.286839 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.286989 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":38}
I20260811 05:08:55.287510 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.278262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60336)
0811 05:08:55.278823 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.278835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.278960 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.279223 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.279225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.279405 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.279536 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.279542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.280488 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.280491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.281320 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.281329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.282478 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.282481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.282563 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.283011 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.283025 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.283402 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.285347 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.285349 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.285351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.285503 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.286125 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.286127 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.286128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.286206 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.286601 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.286604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.286687 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.287213 (+ 526us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.287349 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":76,"threads_started":1}
I20260811 05:08:55.290596 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6088
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424933, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.254@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
I20260811 05:08:55.308594 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.290312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60352)
0811 05:08:55.290541 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.290554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.290624 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.291049 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.291052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.291230 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.291329 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.291335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.292271 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.292275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.293084 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.293094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.293736 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.293739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.293832 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.294279 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.294292 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.296199 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.307116 (+ 10917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.307119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.307121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.307344 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.307626 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.307629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.307631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.307704 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.308137 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.308139 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.308234 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.308316 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.308447 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 05:08:55.309302 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.290510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.30.88.252:44803)
0811 05:08:55.290602 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.290606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.290622 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:08:55.290758 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.290764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.290812 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.290945 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.291460 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.292157 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.293227 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.293546 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.293629 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.296343 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.296347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.296350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.296374 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.307002 (+ 10628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.307461 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.307464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.307466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.307536 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.307789 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.307792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.307794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.307973 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.308081 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.309078 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.309176 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:08:55.310216 6016 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.88.252:44803: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6ffa08b485524c069089c5a668e5f427"
member_type: NON_VOTER
last_known_addr {
host: "127.30.88.252"
port: 44803
}
attrs {
promote: true
}
}
dest_uuid: "df65814267fc41b58ce8a3189c7b4d57"
cas_config_opid_index: 5
I20260811 05:08:55.310282 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.277753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.30.88.253:40321)
0811 05:08:55.277881 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.277886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.278237 (+ 351us) server_negotiation.cc:407] Connection header received
0811 05:08:55.278415 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.278418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.278474 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.278531 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.279174 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.279971 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281099 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.281431 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281500 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.306444 (+ 24944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.306451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.306457 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.306491 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.308311 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.308797 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.308802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.308806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.308899 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.309225 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.309231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.309236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.309470 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.309539 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.309890 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.310028 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:08:55.310822 6016 consensus_queue.cc:237] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } }
I20260811 05:08:55.310952 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.277614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56692)
0811 05:08:55.278157 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.278172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.278299 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.278675 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.278679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.278942 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.279056 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.279062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.280097 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.280100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.280977 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.280985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.281686 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.281690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.281794 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.305881 (+ 24087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.305896 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.306274 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.308440 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.308445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.308449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.308682 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.309016 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.309021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.309025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.309131 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.309609 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.309614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.309721 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.310480 (+ 759us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.310647 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":273,"thread_start_us":109,"threads_started":1}
I20260811 05:08:55.312031 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:46243
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:38967
--webserver_port=40489
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:08:55.312011 6664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6ffa08b485524c069089c5a668e5f427 (127.30.88.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.313414 6664 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.88.252:44803
I20260811 05:08:55.313584 6664 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.313670 6664 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.314008 6824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:55.314131 6671 master.cc:537] Master@127.30.88.252:44803 shutting down...
I20260811 05:08:55.316284 6824 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:08:55.316902 6666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:55.318397 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.312106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56700)
0811 05:08:55.313713 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.313729 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.313803 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.314168 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.314172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.314191 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:55.314262 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.314267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.315365 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.315368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.316835 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.316847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.318017 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.318021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.318112 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.318154 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.318222 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.318283 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514}
I20260811 05:08:55.318717 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.313060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.30.88.253:40321)
0811 05:08:55.313793 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.313797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.313810 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:55.313884 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.313889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.313950 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.314040 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:55.314386 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.315236 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.316973 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.317658 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.317795 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.317957 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.318543 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.318612 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661}
I20260811 05:08:55.319212 6724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 LEARNER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:08:55.319626 6666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 05:08:55.320003 5999 consensus_peers.cc:487] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 -> Peer 6ffa08b485524c069089c5a668e5f427 (127.30.88.252:44803): Couldn't send request to peer 6ffa08b485524c069089c5a668e5f427. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:08:55.321709 6724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6ffa08b485524c069089c5a668e5f427 (127.30.88.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.323300 6671 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:55.323557 6671 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:55.323644 6671 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: stopping tablet replica
I20260811 05:08:55.326246 6883 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.326426 6883 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:08:55.326524 6761 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df65814267fc41b58ce8a3189c7b4d57. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: NON_VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: true } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.326632 6761 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:08:55.327620 6664 raft_consensus.cc:1050] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: attempting to promote NON_VOTER d3f64f2a856d4f23b66e768cf59f84b9 to VOTER
I20260811 05:08:55.328168 6664 consensus_queue.cc:237] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } }
I20260811 05:08:55.329581 6724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 LEARNER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 05:08:55.329977 6884 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 05:08:55.330224 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.329212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60356)
0811 05:08:55.330013 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.330092 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":735}
I20260811 05:08:55.331681 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.331477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60372)
0811 05:08:55.331568 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.331619 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 05:08:55.331837 5999 consensus_peers.cc:487] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 -> Peer 6ffa08b485524c069089c5a668e5f427 (127.30.88.252:44803): Couldn't send request to peer 6ffa08b485524c069089c5a668e5f427. Status: Network error: Client connection negotiation failed: client connection to 127.30.88.252:44803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:08:55.331754 6666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d3f64f2a856d4f23b66e768cf59f84b9 (127.30.88.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.333194 6666 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.333300 6666 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.333622 6664 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.333725 6664 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.333999 6724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d3f64f2a856d4f23b66e768cf59f84b9 (127.30.88.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.335527 6761 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.335634 6761 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:55.338635 6671 master.cc:552] Master@127.30.88.252:44803 shutdown complete.
I20260811 05:08:55.338688 6671 master_runner.cc:305] Clearing existing system tablet
I20260811 05:08:55.338954 6671 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.339008 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.340274 6671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 05:08:55.341091 6887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.341274 6671 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.341392 6671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:08:55.341423 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "6ffa08b485524c069089c5a668e5f427"
format_stamp: "Formatted at 2026-08-11 05:08:55 on dist-test-slave-7c35"
server_key: "68191f77b52d0a2abc5ace1584e497a3"
server_key_iv: "2ddf3f502cde390139e10b0d178855e7"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.343689 6671 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:55.346012 6671 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:08:55.346127 6671 log.cc:1196] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 05:08:55.346382 6671 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Deleting consensus metadata
I20260811 05:08:55.346567 6671 master_runner.cc:315] Copying system tablet from 127.30.88.254:38967
I20260811 05:08:55.347661 6671 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: tablet copy: Beginning tablet copy session from remote peer at address 127.30.88.254:38967
I20260811 05:08:55.356515 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.348141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:58660)
0811 05:08:55.348476 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.348490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.348604 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.348857 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.348860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.349068 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.349212 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.349220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.350257 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.350261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.351150 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.351161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.351785 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.351798 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.351910 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.352521 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.352537 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.352948 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.354799 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.354803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.354805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.355110 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.355416 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.355419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.355421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.355522 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.355945 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.355947 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.356052 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.356171 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.356300 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":87,"threads_started":1}
I20260811 05:08:55.356521 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.348216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.30.88.254:38967)
0811 05:08:55.348323 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.348328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.348580 (+ 252us) server_negotiation.cc:407] Connection header received
0811 05:08:55.348686 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.348689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.348739 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.348819 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.349332 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.350132 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.351276 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.351611 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.351742 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.353101 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.353104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.353107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.353131 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.354669 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.355215 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.355219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.355221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.355315 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.355625 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.355628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.355630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.355841 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.355942 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.356240 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.356352 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:08:55.357414 6037 tablet_copy_service.cc:140] P df65814267fc41b58ce8a3189c7b4d57: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6ffa08b485524c069089c5a668e5f427 ({username='oryx', principal='oryx/127.30.88.252@KRBTEST.COM'} at 127.0.0.1:58660)
I20260811 05:08:55.357511 6037 tablet_copy_service.cc:161] P df65814267fc41b58ce8a3189c7b4d57: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6ffa08b485524c069089c5a668e5f427 at {username='oryx', principal='oryx/127.30.88.252@KRBTEST.COM'} at 127.0.0.1:58660: session id = 6ffa08b485524c069089c5a668e5f427-00000000000000000000000000000000
I20260811 05:08:55.358525 6037 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:08:55.359784 6671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:08:55.361984 6671 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: tablet copy: Starting download of 0 data blocks...
I20260811 05:08:55.362152 6671 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: tablet copy: Starting download of 1 WAL segments...
I20260811 05:08:55.363655 6671 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:08:55.365255 6037 tablet_copy_service.cc:342] P df65814267fc41b58ce8a3189c7b4d57: Request end of tablet copy session 6ffa08b485524c069089c5a668e5f427-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.88.252@KRBTEST.COM'} at 127.0.0.1:58660
I20260811 05:08:55.365335 6037 tablet_copy_service.cc:433] P df65814267fc41b58ce8a3189c7b4d57: ending tablet copy session 6ffa08b485524c069089c5a668e5f427-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6ffa08b485524c069089c5a668e5f427
I20260811 05:08:55.372454 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:55.374102 6900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.374218 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.374733 6671 server_base.cc:1034] running on GCE node
W20260811 05:08:55.374959 6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.375172 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.375552 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.376713 6671 hybrid_clock.cc:648] HybridClock initialized: now 1786424935376698 us; error 29 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.378302 6671 init.cc:377] Logged in from keytab as oryx/127.30.88.252@KRBTEST.COM (short username oryx)
I20260811 05:08:55.378756 6671 webserver.cc:460] Webserver started at http://127.30.88.252:45069/ using document root <none> and password file <none>
I20260811 05:08:55.378973 6671 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.379019 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.380223 6671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:55.381239 6907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.381520 6671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:08:55.381587 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "6ffa08b485524c069089c5a668e5f427"
format_stamp: "Formatted at 2026-08-11 05:08:55 on dist-test-slave-7c35"
server_key: "68191f77b52d0a2abc5ace1584e497a3"
server_key_iv: "2ddf3f502cde390139e10b0d178855e7"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.381654 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.408622 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.411073 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.416157 6671 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.252:44803
I20260811 05:08:55.416293 6974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.252:44803 every 8 connection(s)
I20260811 05:08:55.416728 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
I20260811 05:08:55.418663 6975 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:08:55.419778 6975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Bootstrap starting.
I20260811 05:08:55.430584 6975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:08:55.430879 6975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: Bootstrap complete.
I20260811 05:08:55.431463 6975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } }
I20260811 05:08:55.431595 6975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ffa08b485524c069089c5a668e5f427, State: Initialized, Role: LEARNER
I20260811 05:08:55.431702 6975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } }
I20260811 05:08:55.432013 6978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.432120 6975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:55.432232 6978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:08:55.435485 6989 catalog_manager.cc:1259] Loaded cluster ID: 5943aa5979b84068ac3ef5c80bc8ff36
I20260811 05:08:55.435544 6989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: loading cluster ID for follower catalog manager: success
I20260811 05:08:55.437718 6989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: acquiring CA information for follower catalog manager: success
I20260811 05:08:55.439003 6989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 05:08:55.459086 6881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:55.459303 6881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:55.459388 6881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:55.463356 6881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:55.463402 6881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:55.463460 6881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:55.468263 6881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:46243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=40489
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:55.469115 6881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:55.469970 6881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:55.478945 6997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.478978 6995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.478966 6994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.479066 6881 server_base.cc:1034] running on GCE node
I20260811 05:08:55.480126 6881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.480731 6881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.481056 6392 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
I20260811 05:08:55.481905 6881 hybrid_clock.cc:648] HybridClock initialized: now 1786424935481888 us; error 43 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.484575 6881 init.cc:377] Logged in from keytab as oryx/127.30.88.193@KRBTEST.COM (short username oryx)
I20260811 05:08:55.485427 6881 webserver.cc:460] Webserver started at http://127.30.88.193:40489/ using document root <none> and password file <none>
I20260811 05:08:55.485970 6881 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.486083 6881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.489647 6881 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:55.491573 7004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.492789 6881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:55.492923 6881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "58f06f5b075943c29216783a19ae558a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "0040ff494b88d45cfa97f7116c5cc180"
server_key_iv: "50bc40d49ad571adf9747e2b40546d04"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.493410 6881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.509631 6881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.512635 6881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:55.512818 6881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.513326 6881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:55.514075 6881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:55.514140 6881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.514201 6881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:55.514246 6881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.523900 6881 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:46243
I20260811 05:08:55.523947 7140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:46243 every 8 connection(s)
I20260811 05:08:55.524865 6881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:55.533397 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 6881
I20260811 05:08:55.533618 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6242
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
I20260811 05:08:55.541962 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.528103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.30.88.193:47589)
0811 05:08:55.528392 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.528405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.528694 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.529398 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.529400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.529786 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.530427 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.530442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531844 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.531847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.532667 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.532674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.533486 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.534157 (+ 665us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.534595 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.534616 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.536430 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.538331 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.538339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.538354 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.538693 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.539050 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.539055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.539057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.539227 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.539800 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.539809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.540270 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.540483 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.540814 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":91,"spinlock_wait_cycles":10240,"thread_start_us":113,"threads_started":1}
I20260811 05:08:55.542701 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.526621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:36359 (local address 127.30.88.254:38967)
0811 05:08:55.526749 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.526754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.527990 (+ 1236us) server_negotiation.cc:407] Connection header received
0811 05:08:55.529097 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.529103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.529163 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.529242 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.530606 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531223 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.532642 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.532971 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533067 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.538234 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.538241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.538247 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.538285 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.540127 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.540760 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.540767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.540771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.540887 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541344 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.541350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.541354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.541657 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.541748 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.542280 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.542448 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:08:55.542846 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.528257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:47589 (local address 127.30.88.252:44803)
0811 05:08:55.528474 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.528489 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.528502 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:55.529234 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.529239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.529297 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.529383 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.530596 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531716 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.532800 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.533231 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533332 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.536619 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.536622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.536624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.536646 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.538212 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.538810 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.538815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.538817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.538929 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.539338 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.539344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.539348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.539646 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.539786 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.542512 (+ 2726us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.542664 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260811 05:08:55.543426 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.526620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:48591 (local address 127.30.88.253:40321)
0811 05:08:55.526748 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.526753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.527747 (+ 994us) server_negotiation.cc:407] Connection header received
0811 05:08:55.528960 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.528964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.529036 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.529103 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.530598 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531467 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.532581 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.532900 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533028 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.538119 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.538124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.538127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.538154 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.541027 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541544 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.541547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.541549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.541634 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541954 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.541960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.541961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.542199 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.542276 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.542665 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.542788 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:08:55.543885 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.527081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.30.88.193:48591)
0811 05:08:55.527570 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.527621 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.528692 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.529297 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.529306 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.529782 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.530427 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.530442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531607 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.531610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.532466 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.532476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.533025 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.533225 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.535704 (+ 2479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.535724 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.537956 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.541176 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.541181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.541417 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.541743 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.541748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.541850 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.542354 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.542360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.542465 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.543576 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.543723 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":86,"spinlock_wait_cycles":19200,"thread_start_us":560,"threads_started":1}
I20260811 05:08:55.543901 7141 heartbeater.cc:347] Connected to a master server at 127.30.88.252:44803
I20260811 05:08:55.544201 7141 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.544935 7141 heartbeater.cc:511] Master 127.30.88.252:44803 requested a full tablet report, sending...
I20260811 05:08:55.546018 7143 heartbeater.cc:347] Connected to a master server at 127.30.88.253:40321
I20260811 05:08:55.546108 7143 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.546170 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.527081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:36359)
0811 05:08:55.527890 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.527904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.528751 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.529298 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.529307 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.529831 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.530425 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.530450 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.531364 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.531368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.532529 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.532540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.533184 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.533189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.533316 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.535346 (+ 2030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.535368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.538050 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.540263 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.540269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.540274 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.540603 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.541043 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.541049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.541054 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.541190 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.541860 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.541867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.542012 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.543027 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.543232 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":114,"spinlock_wait_cycles":16896,"thread_start_us":138,"threads_started":1}
I20260811 05:08:55.546259 7143 heartbeater.cc:511] Master 127.30.88.253:40321 requested a full tablet report, sending...
I20260811 05:08:55.546746 6928 ts_manager.cc:194] Registered new tserver with Master: 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:55.547153 7142 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
I20260811 05:08:55.547230 7142 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.547292 6710 ts_manager.cc:194] Registered new tserver with Master: 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:55.547380 7142 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
I20260811 05:08:55.548100 6016 ts_manager.cc:194] Re-registered known tserver with Master: 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:55.548635 6016 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:36359
I20260811 05:08:55.557590 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:46527
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:38967
--webserver_port=40935
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:55.676206 7149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:55.676424 7149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:55.676462 7149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:55.680285 7149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:55.680331 7149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:55.680433 7149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:55.685421 7149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:46527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=40935
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:55.686443 7149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:55.687426 7149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:08:55.689306 6548 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
W20260811 05:08:55.696851 7156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.696915 7160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.696928 7149 server_base.cc:1034] running on GCE node
W20260811 05:08:55.696846 7158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.697548 7149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.698125 7149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.699318 7149 hybrid_clock.cc:648] HybridClock initialized: now 1786424935699307 us; error 26 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.702088 7149 init.cc:377] Logged in from keytab as oryx/127.30.88.194@KRBTEST.COM (short username oryx)
I20260811 05:08:55.703022 7149 webserver.cc:460] Webserver started at http://127.30.88.194:40935/ using document root <none> and password file <none>
I20260811 05:08:55.703526 7149 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.703613 7149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.707405 7149 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 05:08:55.709492 7169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.710830 7149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:55.711031 7149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "912d16ff05c64cb49f23321f33dc971a"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "28ff7c6c80c8f896446b2fc952216d96"
server_key_iv: "e77aa6baf43249729bb4cd1b5bcadd8b"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.711570 7149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.723320 7149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.725947 7149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:55.726151 7149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.726701 7149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:55.727620 7149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:55.727695 7149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.727767 7149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:55.727810 7149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.737156 7149 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:46527
I20260811 05:08:55.737234 7295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:46527 every 8 connection(s)
I20260811 05:08:55.738198 7149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:55.744335 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 7149
I20260811 05:08:55.744503 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6397
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
I20260811 05:08:55.756255 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:39027
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:38967
--webserver_port=34389
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:08:55.756919 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.741930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:33483 (local address 127.30.88.252:44803)
0811 05:08:55.742083 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.742088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.742103 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:08:55.742432 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.742436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.742499 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.742550 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.743937 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744719 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.746742 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747113 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747189 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.752120 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.752125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.752130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.752167 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.754222 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.754997 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755093 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755526 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755764 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.755847 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.756611 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.756755 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:08:55.756918 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.740002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:47079 (local address 127.30.88.254:38967)
0811 05:08:55.740171 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.740176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.741506 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 05:08:55.742433 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.742436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.742494 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.742550 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.743936 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744715 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.746742 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747111 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747189 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.751593 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.751600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.751606 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.751641 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.754498 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755193 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755305 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755633 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755840 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.755917 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.756644 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.756753 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:08:55.757043 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.740406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:36501 (local address 127.30.88.253:40321)
0811 05:08:55.740551 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.740557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.741277 (+ 720us) server_negotiation.cc:407] Connection header received
0811 05:08:55.742415 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.742419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.742480 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.742558 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.743917 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744647 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.746772 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747205 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747283 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.752722 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.752726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.752729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.752764 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.754450 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755162 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755266 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755612 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.755618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.755623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.755867 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.755953 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.756666 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.756807 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:08:55.758766 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.741548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.30.88.194:33483)
0811 05:08:55.741777 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.741811 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.742228 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.742728 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.742731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.743239 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.743789 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.743796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744961 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.744964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.746608 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.746615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747390 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.747915 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.748775 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.748800 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.751931 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.754380 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.754387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.754404 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.754872 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.755208 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.755212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.755404 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.755919 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.755928 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.756293 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.757228 (+ 935us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.757743 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":932,"spinlock_wait_cycles":72576,"thread_start_us":78,"threads_started":1}
I20260811 05:08:55.758766 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.740403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:47079)
0811 05:08:55.741408 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.741419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.742228 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.742728 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.742731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.743240 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.743787 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.743796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744883 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.744886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.746596 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.746608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747390 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.747876 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.748818 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.748830 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.751385 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.754636 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.754639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.754641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.755073 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.755423 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.755432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.755529 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.755978 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.755981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.756293 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.757307 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.757685 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":660,"spinlock_wait_cycles":128512,"thread_start_us":155,"threads_started":1}
I20260811 05:08:55.759415 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.740402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.30.88.194:36501)
0811 05:08:55.741082 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.741144 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.742207 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.742722 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.742728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.743237 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.743776 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.743788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.744845 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.744851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.746628 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.746643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.747464 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.747471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.747910 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.748962 (+ 1052us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.748982 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.752598 (+ 3616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.754595 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.754601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.754605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.755039 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.755389 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.755392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.755394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.755503 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.756017 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.756021 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.756293 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.757495 (+ 1202us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.759227 (+ 1732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":2008,"spinlock_wait_cycles":5248,"thread_start_us":743,"threads_started":1}
I20260811 05:08:55.760087 7298 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
I20260811 05:08:55.760169 7299 heartbeater.cc:347] Connected to a master server at 127.30.88.253:40321
I20260811 05:08:55.760313 7299 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.760337 7297 heartbeater.cc:347] Connected to a master server at 127.30.88.252:44803
I20260811 05:08:55.760398 7297 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.760825 7297 heartbeater.cc:511] Master 127.30.88.252:44803 requested a full tablet report, sending...
I20260811 05:08:55.760313 7298 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.761135 7298 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
I20260811 05:08:55.760828 7299 heartbeater.cc:511] Master 127.30.88.253:40321 requested a full tablet report, sending...
I20260811 05:08:55.761817 6016 ts_manager.cc:194] Re-registered known tserver with Master: 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527)
I20260811 05:08:55.761881 6928 ts_manager.cc:194] Registered new tserver with Master: 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527)
I20260811 05:08:55.762661 6016 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:47079
I20260811 05:08:55.763548 6710 ts_manager.cc:194] Registered new tserver with Master: 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527)
W20260811 05:08:55.870759 7304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:55.871011 7304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:55.871050 7304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:55.874927 7304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:55.874974 7304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:55.875027 7304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:08:55.879806 7304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:39027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=34389
--tserver_master_addrs=127.30.88.254:38967,127.30.88.253:40321,127.30.88.252:44803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:55.880756 7304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:55.881769 7304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:55.890873 7309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.890985 7304 server_base.cc:1034] running on GCE node
W20260811 05:08:55.890832 7312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:55.891189 7310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:55.891646 7304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:55.892194 7304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:55.893368 7304 hybrid_clock.cc:648] HybridClock initialized: now 1786424935893330 us; error 45 us; skew 500 ppm
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:55.895985 7304 init.cc:377] Logged in from keytab as oryx/127.30.88.195@KRBTEST.COM (short username oryx)
I20260811 05:08:55.896853 7304 webserver.cc:460] Webserver started at http://127.30.88.195:34389/ using document root <none> and password file <none>
I20260811 05:08:55.897361 7304 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:55.897480 7304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:55.900957 7304 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:08:55.902740 7321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.903951 7304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:55.904086 7304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "88bd0b78ec53475980c1f8b9752250a4"
format_stamp: "Formatted at 2026-08-11 05:08:54 on dist-test-slave-7c35"
server_key: "bdef36f2e24ae540d4e538fc8cf236c4"
server_key_iv: "956700d086a147c390b0c4d56178125e"
server_key_version: "encryptionkey@0"
I20260811 05:08:55.904563 7304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:55.911800 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.908059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60386)
0811 05:08:55.908285 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.908301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.908393 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.908685 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.908687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.908701 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:55.908789 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.908793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.909813 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.909816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.910576 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.910585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.911466 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.911468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.911539 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.911561 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.911624 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.911693 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260811 05:08:55.911811 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.908178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.30.88.252:44803)
0811 05:08:55.908310 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.908314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.908370 (+ 56us) server_negotiation.cc:407] Connection header received
0811 05:08:55.908495 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.908498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.908568 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.908671 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:55.908892 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.909686 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.910683 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.911360 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.911458 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.911580 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.911636 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.911703 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:08:55.912413 6944 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 05:08:55.912762 6978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df65814267fc41b58ce8a3189c7b4d57. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.912917 6978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:08:55.919552 7304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:55.922256 7304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:55.922446 7304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:55.922984 7304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:08:55.923729 7304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:55.923794 7304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.923856 7304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:55.923903 7304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:55.931746 7304 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39027
I20260811 05:08:55.931766 7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39027 every 8 connection(s)
I20260811 05:08:55.932652 7304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:08:55.933643 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 7304
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.253@KRBTEST.COM
I20260811 05:08:55.947052 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.936003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.30.88.254:38967)
0811 05:08:55.936712 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.936716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.936727 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:08:55.936766 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.936771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.936837 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.936927 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.938578 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.939296 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.940404 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940726 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.940806 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.942615 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.942619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.942621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.942646 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.945130 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.945596 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.945600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.945602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.945686 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.945994 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.945998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.945999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.946205 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.946303 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.946660 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.946797 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":100,"threads_started":1}
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:55.948681 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.936683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.30.88.252:44803)
0811 05:08:55.938805 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.938811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.939037 (+ 226us) server_negotiation.cc:407] Connection header received
0811 05:08:55.939197 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.939199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.939244 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.939299 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.940159 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940874 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.942319 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.942674 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.942740 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.944498 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.944502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.944505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.944530 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.946501 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.947044 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.947050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.947052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.947389 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.947720 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.947741 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.947743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.947934 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.948038 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.948365 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.948483 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2060,"thread_start_us":80,"threads_started":1}
I20260811 05:08:55.952098 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.934600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:48395 (local address 127.30.88.253:40321)
0811 05:08:55.934812 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.934816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.936581 (+ 1765us) server_negotiation.cc:407] Connection header received
0811 05:08:55.938268 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.938273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.938315 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.938376 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.939972 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940661 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.942594 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.942980 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.943068 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.946505 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.946508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.946511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.946538 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.948323 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.949179 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.949185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.949187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.949435 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.949835 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.949839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.949840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.951043 (+ 1203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.951123 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.951827 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.951941 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 05:08:55.953397 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.934958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:34321 (local address 127.30.88.254:38967)
0811 05:08:55.935373 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.935378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.936581 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 05:08:55.938200 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.938203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.938245 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.938289 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.939974 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940721 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.942535 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.942900 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.942968 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.949737 (+ 6769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.949740 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.949742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.949764 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.951627 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.952057 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.952060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.952062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.952132 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.952645 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.952648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.952650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.952818 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.952884 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.953174 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.953271 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334}
I20260811 05:08:55.950708 6016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:08:55.954101 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.934949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:34321)
0811 05:08:55.936452 (+ 1503us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.936500 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.937984 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.938556 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.938564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.939115 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.939831 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.939847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940875 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.940879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.942417 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.942426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.943160 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.943167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.943343 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.946254 (+ 2911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.946270 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.949592 (+ 3322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.951741 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.951744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.951746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.951955 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.952465 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.952468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.952470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.952549 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.952936 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.952939 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.953015 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.953528 (+ 513us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.953653 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"mutex_wait_us":113,"spinlock_wait_cycles":20096,"thread_start_us":140,"threads_started":1}
I20260811 05:08:55.954936 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.934949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.30.88.195:48395)
0811 05:08:55.936491 (+ 1542us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.936502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.937990 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.938557 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.938565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.939112 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.939827 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.939844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.940850 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.940853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.942434 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.942443 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.943231 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.943235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.943352 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.944118 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.944143 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.946046 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.948466 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.948472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.948485 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.948828 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.949561 (+ 733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.949564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.949566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.949722 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.951217 (+ 1495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.951226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.951555 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.952458 (+ 903us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.952797 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1472,"mutex_wait_us":239,"spinlock_wait_cycles":37248,"thread_start_us":145,"threads_started":1}
I20260811 05:08:55.955226 7451 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38967
W20260811 05:08:55.955230 6016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:55.955468 7451 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.955688 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.935729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.30.88.253:40321)
0811 05:08:55.936358 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.936363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.937311 (+ 948us) server_negotiation.cc:407] Connection header received
0811 05:08:55.937403 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.937405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.937458 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.937546 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.947462 (+ 9916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.948230 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.950095 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.950353 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.950466 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.951941 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.951944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.951946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.951965 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.953521 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.953979 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.953983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.953986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.954082 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.954646 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.954651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.954654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.954903 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.955031 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.955395 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.955515 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":122,"threads_started":1}
I20260811 05:08:55.956027 7451 heartbeater.cc:511] Master 127.30.88.254:38967 requested a full tablet report, sending...
I20260811 05:08:55.956032 7453 heartbeater.cc:347] Connected to a master server at 127.30.88.253:40321
I20260811 05:08:55.956501 7453 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.956666 7453 heartbeater.cc:511] Master 127.30.88.253:40321 requested a full tablet report, sending...
I20260811 05:08:55.957127 6015 ts_manager.cc:194] Re-registered known tserver with Master: 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
Aug 11 05:08:55 dist-test-slave-7c35 krb5kdc[5962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.252@KRBTEST.COM
I20260811 05:08:55.957677 6707 ts_manager.cc:194] Registered new tserver with Master: 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
I20260811 05:08:55.957799 6015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:34321
I20260811 05:08:55.959285 7322 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: NON_VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: true } } }
I20260811 05:08:55.959405 7322 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.961531 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.935978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.30.88.195:46893)
0811 05:08:55.936618 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.936645 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.937984 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.938556 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.938564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.943765 (+ 5201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:55.943899 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.943908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.945007 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.945010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.945865 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.945875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.953700 (+ 7825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.953706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.953804 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.954428 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:55.954441 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:55.958041 (+ 3600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:55.959961 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.959963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.959965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.960222 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.960534 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:55.960537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:55.960539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:55.960630 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:55.961044 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:55.961049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:55.961136 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.961239 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.961370 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"spinlock_wait_cycles":11904,"thread_start_us":92,"threads_started":1}
I20260811 05:08:55.961560 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.936350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:46893 (local address 127.30.88.252:44803)
0811 05:08:55.936460 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.936464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.936705 (+ 241us) server_negotiation.cc:407] Connection header received
0811 05:08:55.938200 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.938203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.938245 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.938289 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:55.944045 (+ 5756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.944854 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.945993 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.946301 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.946371 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.958181 (+ 11810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:55.958185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:55.958188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:55.958211 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:55.959857 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.960326 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.960332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.960334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.960431 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:55.960731 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:55.960734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:55.960736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:55.960934 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:55.961031 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.961305 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.961427 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:08:55.962704 7450 heartbeater.cc:347] Connected to a master server at 127.30.88.252:44803
I20260811 05:08:55.962785 7450 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:55.962965 7450 heartbeater.cc:511] Master 127.30.88.252:44803 requested a full tablet report, sending...
I20260811 05:08:55.963377 7474 raft_consensus.cc:1050] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: attempting to promote NON_VOTER 6ffa08b485524c069089c5a668e5f427 to VOTER
I20260811 05:08:55.963725 6928 ts_manager.cc:194] Registered new tserver with Master: 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
I20260811 05:08:55.963994 7474 consensus_queue.cc:237] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } }
I20260811 05:08:55.965281 6944 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 LEARNER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 05:08:55.965682 6724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Refusing update from remote peer df65814267fc41b58ce8a3189c7b4d57: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 05:08:55.965687 7322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:08:55.966017 7322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:08:55.967587 7474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6ffa08b485524c069089c5a668e5f427 (127.30.88.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.968075 6724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6ffa08b485524c069089c5a668e5f427 (127.30.88.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.969084 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.969183 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.969023 6944 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6ffa08b485524c069089c5a668e5f427 (127.30.88.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.969312 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.969389 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.969481 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.969547 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:55.970799 6978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.970930 6978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:55.975615 7485 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df65814267fc41b58ce8a3189c7b4d57" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:55.975741 7485 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:55.978768 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.973672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46243 (local address 127.0.0.1:52132)
0811 05:08:55.973815 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.973827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.973911 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.975387 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.975390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.975406 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:55.975513 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.975520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.976819 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.976824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.977673 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.977682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978411 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.978415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.978499 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.978518 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.978591 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.978655 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":44}
I20260811 05:08:55.978828 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.974316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.30.88.193:46243)
0811 05:08:55.974899 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.974919 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.974945 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:08:55.975032 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.975038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.975241 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.975405 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:55.975645 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.976720 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.977783 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978294 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.978402 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.978540 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.978620 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.978705 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":96,"threads_started":1}
I20260811 05:08:55.979943 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.974180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:46527 (local address 127.0.0.1:34500)
0811 05:08:55.974962 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.974976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.975070 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.975690 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.975695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.975713 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:55.975785 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.975791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.976668 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.976672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.977677 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.977688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978816 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.978819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.979641 (+ 822us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.979664 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.979730 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.979814 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":5,"thread_start_us":95,"threads_started":1}
I20260811 05:08:55.980479 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.974352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34500 (local address 127.30.88.194:46527)
0811 05:08:55.975077 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.975089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.975118 (+ 29us) server_negotiation.cc:407] Connection header received
0811 05:08:55.975192 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.975197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.975371 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.975528 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:55.975900 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.976555 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.977810 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978552 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.978648 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.978762 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.980184 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.980291 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":111,"threads_started":1}
I20260811 05:08:55.980888 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.974173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39027 (local address 127.0.0.1:44804)
0811 05:08:55.974611 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:55.974641 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:55.974743 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:55.976848 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:55.976852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:55.976870 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:55.977001 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.977006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978196 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.978199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.979388 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:55.979398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.980365 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.980368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:55.980493 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.980516 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:08:55.980709 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 05:08:55.980776 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":17,"thread_start_us":98,"threads_started":1}
I20260811 05:08:55.980955 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:55.974578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.30.88.195:39027)
0811 05:08:55.974950 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:55.974962 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:55.974980 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:08:55.975033 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:55.975038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:55.975241 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:55.975405 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:55.977131 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.978084 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.979532 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:55.980251 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:55.980348 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:55.980476 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:55.980752 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 05:08:55.980829 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":139,"threads_started":1}
I20260811 05:08:55.981768 7061 tablet_service.cc:1467] Processing CreateTablet for tablet f9eea4b6b71540fdbbffac31e873015a (DEFAULT_TABLE table=test-table [id=e6e5921475a949bebe26b5f32f616d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:55.982734 7061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9eea4b6b71540fdbbffac31e873015a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:55.982658 7219 tablet_service.cc:1467] Processing CreateTablet for tablet f9eea4b6b71540fdbbffac31e873015a (DEFAULT_TABLE table=test-table [id=e6e5921475a949bebe26b5f32f616d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:55.983636 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9eea4b6b71540fdbbffac31e873015a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:55.985154 7374 tablet_service.cc:1467] Processing CreateTablet for tablet f9eea4b6b71540fdbbffac31e873015a (DEFAULT_TABLE table=test-table [id=e6e5921475a949bebe26b5f32f616d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:55.986100 7374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9eea4b6b71540fdbbffac31e873015a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:55.989073 7492 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Bootstrap starting.
I20260811 05:08:55.990779 7493 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Bootstrap starting.
I20260811 05:08:55.991276 7492 tablet_bootstrap.cc:654] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:55.992162 7492 log.cc:824] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:55.992902 7493 tablet_bootstrap.cc:654] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:55.993641 7493 log.cc:824] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:55.994113 7492 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: No bootstrap required, opened a new log
I20260811 05:08:55.994266 7492 ts_tablet_manager.cc:1397] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:08:55.995220 7495 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Bootstrap starting.
I20260811 05:08:55.995463 7493 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: No bootstrap required, opened a new log
I20260811 05:08:55.995659 7493 ts_tablet_manager.cc:1397] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:08:55.996869 7495 tablet_bootstrap.cc:654] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:55.997936 7495 log.cc:824] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:55.998569 7492 raft_consensus.cc:348] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:55.998840 7492 raft_consensus.cc:374] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:55.998921 7492 raft_consensus.cc:729] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f06f5b075943c29216783a19ae558a, State: Initialized, Role: FOLLOWER
I20260811 05:08:55.999325 7492 consensus_queue.cc:260] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:55.999646 7495 tablet_bootstrap.cc:492] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: No bootstrap required, opened a new log
I20260811 05:08:55.999794 7495 ts_tablet_manager.cc:1397] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:08:55.999914 7493 raft_consensus.cc:348] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.000114 7493 raft_consensus.cc:374] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:56.000156 7493 raft_consensus.cc:729] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912d16ff05c64cb49f23321f33dc971a, State: Initialized, Role: FOLLOWER
I20260811 05:08:56.000226 7492 ts_tablet_manager.cc:1428] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:08:56.000566 7493 consensus_queue.cc:260] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.001446 7493 ts_tablet_manager.cc:1428] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:08:56.001456 7142 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
I20260811 05:08:56.002278 7298 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
I20260811 05:08:56.008848 7495 raft_consensus.cc:348] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.009049 7495 raft_consensus.cc:374] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:56.009125 7495 raft_consensus.cc:729] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bd0b78ec53475980c1f8b9752250a4, State: Initialized, Role: FOLLOWER
I20260811 05:08:56.009480 7495 consensus_queue.cc:260] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.010056 7451 heartbeater.cc:503] Master 127.30.88.254:38967 was elected leader, sending a full tablet report...
I20260811 05:08:56.010344 7495 ts_tablet_manager.cc:1428] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
W20260811 05:08:56.026316 7144 tablet.cc:2369] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:08:56.183915 7454 tablet.cc:2369] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:56.197377 7499 raft_consensus.cc:482] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:56.197630 7499 raft_consensus.cc:504] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.198828 7499 leader_election.cc:288] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243), 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
I20260811 05:08:56.202432 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.199171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:41559 (local address 127.30.88.193:46243)
0811 05:08:56.199301 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.199305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.199358 (+ 53us) server_negotiation.cc:407] Connection header received
0811 05:08:56.199526 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.199529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.199581 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.199661 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:56.199903 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.200373 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.201427 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.201841 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.201968 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.202020 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.202223 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.202298 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:08:56.202441 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.199289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:36275 (local address 127.30.88.195:39027)
0811 05:08:56.199420 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.199424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.199438 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:56.199575 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.199578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.199623 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.199702 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:56.199966 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.200413 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.201443 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.201872 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.202012 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.202060 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.202249 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.202315 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:08:56.202459 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.199080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46243 (local address 127.30.88.194:41559)
0811 05:08:56.199277 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:56.199291 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:56.199402 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:56.199672 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:56.199675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:56.199689 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:56.199800 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:56.199805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.200498 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.200504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:56.201313 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:56.201321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.201960 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.201964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:56.202067 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.202118 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:08:56.202204 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 05:08:56.202298 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":63}
I20260811 05:08:56.202472 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.199171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39027 (local address 127.30.88.194:36275)
0811 05:08:56.199344 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:56.199358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:56.199458 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:56.199713 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:56.199715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:56.199731 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:56.199849 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:56.199854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.200545 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.200549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:56.201325 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:56.201334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.202005 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.202008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:56.202120 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.202148 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:08:56.202232 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:08:56.202312 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":35}
I20260811 05:08:56.203097 7397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9eea4b6b71540fdbbffac31e873015a" candidate_uuid: "912d16ff05c64cb49f23321f33dc971a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bd0b78ec53475980c1f8b9752250a4" is_pre_election: true
I20260811 05:08:56.203112 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9eea4b6b71540fdbbffac31e873015a" candidate_uuid: "912d16ff05c64cb49f23321f33dc971a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f06f5b075943c29216783a19ae558a" is_pre_election: true
I20260811 05:08:56.203366 7397 raft_consensus.cc:2454] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 912d16ff05c64cb49f23321f33dc971a in term 0.
I20260811 05:08:56.203395 7087 raft_consensus.cc:2454] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 912d16ff05c64cb49f23321f33dc971a in term 0.
I20260811 05:08:56.203841 7171 leader_election.cc:302] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88bd0b78ec53475980c1f8b9752250a4, 912d16ff05c64cb49f23321f33dc971a; no voters:
I20260811 05:08:56.204103 7499 raft_consensus.cc:2781] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:56.204205 7499 raft_consensus.cc:482] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:56.204259 7499 raft_consensus.cc:3037] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:56.205330 7499 raft_consensus.cc:504] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.205696 7499 leader_election.cc:288] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [CANDIDATE]: Term 1 election: Requested vote from peers 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243), 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027)
I20260811 05:08:56.205971 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9eea4b6b71540fdbbffac31e873015a" candidate_uuid: "912d16ff05c64cb49f23321f33dc971a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f06f5b075943c29216783a19ae558a"
I20260811 05:08:56.206107 7087 raft_consensus.cc:3037] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:56.206156 7397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9eea4b6b71540fdbbffac31e873015a" candidate_uuid: "912d16ff05c64cb49f23321f33dc971a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bd0b78ec53475980c1f8b9752250a4"
I20260811 05:08:56.206281 7397 raft_consensus.cc:3037] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:56.207212 7087 raft_consensus.cc:2454] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 912d16ff05c64cb49f23321f33dc971a in term 1.
I20260811 05:08:56.207481 7172 leader_election.cc:302] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f06f5b075943c29216783a19ae558a, 912d16ff05c64cb49f23321f33dc971a; no voters:
I20260811 05:08:56.207635 7499 raft_consensus.cc:2781] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:56.207614 7397 raft_consensus.cc:2454] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 912d16ff05c64cb49f23321f33dc971a in term 1.
I20260811 05:08:56.207857 7499 raft_consensus.cc:686] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 1 LEADER]: Becoming Leader. State: Replica: 912d16ff05c64cb49f23321f33dc971a, State: Running, Role: LEADER
I20260811 05:08:56.208374 7499 consensus_queue.cc:237] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } }
I20260811 05:08:56.212776 6014 catalog_manager.cc:5495] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a reported cstate change: term changed from 0 to 1, leader changed from <none> to 912d16ff05c64cb49f23321f33dc971a (127.30.88.194). New cstate: current_term: 1 leader_uuid: "912d16ff05c64cb49f23321f33dc971a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } health_report { overall_health: HEALTHY } } }
W20260811 05:08:56.239843 7300 tablet.cc:2369] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:08:56.271369 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.268272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.30.88.194:46527)
0811 05:08:56.268384 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.268388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.268400 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:08:56.268495 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.268497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.268546 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.268627 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:56.268821 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.269311 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.270113 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.270330 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.270403 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.270648 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:56.271057 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:56.271144 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.271212 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.271257 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:08:56.277119 7087 raft_consensus.cc:1261] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Refusing update from remote peer 912d16ff05c64cb49f23321f33dc971a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:56.277220 7397 raft_consensus.cc:1261] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Refusing update from remote peer 912d16ff05c64cb49f23321f33dc971a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:56.277884 7501 consensus_queue.cc:1035] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:56.278031 7499 consensus_queue.cc:1035] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:56.291450 7505 mvcc.cc:205] Tried to move back new op lower bound from 7317196538983796736 to 7317196538712670208. Current Snapshot: MvccSnapshot[applied={T|T < 7317196538983796736 or (T in {7317196538983796736})}]
I20260811 05:08:56.292799 7508 mvcc.cc:205] Tried to move back new op lower bound from 7317196538983796736 to 7317196538712670208. Current Snapshot: MvccSnapshot[applied={T|T < 7317196538983796736 or (T in {7317196538983796736})}]
I20260811 05:08:56.295825 7506 mvcc.cc:205] Tried to move back new op lower bound from 7317196538983796736 to 7317196538712670208. Current Snapshot: MvccSnapshot[applied={T|T < 7317196538983796736 or (T in {7317196538983796736})}]
I20260811 05:08:56.298025 6015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:56.298189 6015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:56.301976 6015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df65814267fc41b58ce8a3189c7b4d57: Sending DeleteTablet for 3 replicas of tablet f9eea4b6b71540fdbbffac31e873015a
I20260811 05:08:56.302826 7061 tablet_service.cc:1514] Processing DeleteTablet for tablet f9eea4b6b71540fdbbffac31e873015a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:56 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:52132
I20260811 05:08:56.302829 7374 tablet_service.cc:1514] Processing DeleteTablet for tablet f9eea4b6b71540fdbbffac31e873015a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:56 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44804
I20260811 05:08:56.303617 7219 tablet_service.cc:1514] Processing DeleteTablet for tablet f9eea4b6b71540fdbbffac31e873015a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:56 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:34500
I20260811 05:08:56.304065 7514 tablet_replica.cc:330] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: stopping tablet replica
I20260811 05:08:56.304354 7514 raft_consensus.cc:2229] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:56.304634 7514 raft_consensus.cc:2258] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:56.305071 7517 tablet_replica.cc:330] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: stopping tablet replica
I20260811 05:08:56.305327 7517 raft_consensus.cc:2229] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:56.305630 7517 raft_consensus.cc:2258] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:56.306782 7514 ts_tablet_manager.cc:1905] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:56.306825 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 5978
I20260811 05:08:56.311743 7517 ts_tablet_manager.cc:1905] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:56.314404 7517 ts_tablet_manager.cc:1918] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:56.314504 7517 log.cc:1196] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/f9eea4b6b71540fdbbffac31e873015a
I20260811 05:08:56.315840 7515 tablet_replica.cc:330] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: stopping tablet replica
I20260811 05:08:56.316147 7515 raft_consensus.cc:2229] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:56.316457 7515 raft_consensus.cc:2258] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:56.317441 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.304271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.30.88.253:40321)
0811 05:08:56.304371 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.304376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.304390 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:56.304429 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.304432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.304478 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.304566 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:56.306390 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.307124 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.310417 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.310835 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.310946 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.313130 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:56.313137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:56.313144 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:56.313184 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:56.315017 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.315681 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.315687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.315692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.315785 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.316191 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.316197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.316201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.316461 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:56.316585 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.317077 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.317228 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 05:08:56.318162 7514 ts_tablet_manager.cc:1918] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:56.318254 7514 log.cc:1196] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/f9eea4b6b71540fdbbffac31e873015a
I20260811 05:08:56.318589 7514 ts_tablet_manager.cc:1939] T f9eea4b6b71540fdbbffac31e873015a P 88bd0b78ec53475980c1f8b9752250a4: Deleting consensus metadata
I20260811 05:08:56.318810 7515 ts_tablet_manager.cc:1905] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:56.320322 7517 ts_tablet_manager.cc:1939] T f9eea4b6b71540fdbbffac31e873015a P 912d16ff05c64cb49f23321f33dc971a: Deleting consensus metadata
W20260811 05:08:56.321291 7172 connection.cc:441] server connection from 127.0.0.1:34500 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34500 (request call id 1) could send its response
I20260811 05:08:56.321532 7515 ts_tablet_manager.cc:1918] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:56.321645 7515 log.cc:1196] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/f9eea4b6b71540fdbbffac31e873015a
I20260811 05:08:56.321928 7515 ts_tablet_manager.cc:1939] T f9eea4b6b71540fdbbffac31e873015a P 58f06f5b075943c29216783a19ae558a: Deleting consensus metadata
W20260811 05:08:56.322896 7008 connection.cc:441] server connection from 127.0.0.1:52132 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52132 (request call id 1) could send its response
I20260811 05:08:56.327566 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.304371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.30.88.252:44803)
0811 05:08:56.304470 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.304474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.318203 (+ 13729us) server_negotiation.cc:407] Connection header received
0811 05:08:56.318472 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.318475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.318538 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.318587 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:56.319636 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.320440 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.321672 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.321978 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.322053 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.323662 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:56.323668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:56.323672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:56.323702 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:56.325422 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.325964 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.325967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.325969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.326057 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.326355 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.326358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.326360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.326556 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:56.326671 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.327314 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.327417 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:08:56.333141 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.324491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.30.88.252:44803)
0811 05:08:56.324586 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.324592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.325030 (+ 438us) server_negotiation.cc:407] Connection header received
0811 05:08:56.325212 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.325217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.325265 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.325320 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:56.326182 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.326864 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.327966 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.328266 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.328355 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.329669 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:56.329672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:56.329675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:56.329695 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:56.331458 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.331840 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.331843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.331845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.331939 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.332235 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.332238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.332240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.332434 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:56.332560 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.332862 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.332984 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:08:56.334175 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:56.323847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.30.88.253:40321)
0811 05:08:56.323968 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:56.323972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:56.324884 (+ 912us) server_negotiation.cc:407] Connection header received
0811 05:08:56.325048 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:56.325051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:56.325091 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:56.325137 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:56.326040 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.326797 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.328140 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:56.328491 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:56.328562 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:56.330379 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:56.330384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:56.330389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:56.330419 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:56.332215 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.332663 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.332667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.332668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.332792 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:56.333200 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:56.333203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:56.333205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:56.333421 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:56.333521 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:56.333900 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 05:08:56.334000 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:57.291105 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.290417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:44069)
0811 05:08:57.290856 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.290984 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":158,"threads_started":1}
W20260811 05:08:57.291484 7142 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:38967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.298830 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.298456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:39529)
0811 05:08:57.298683 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.298750 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260811 05:08:57.298915 7526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.298497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:50141)
0811 05:08:57.298713 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.298787 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
W20260811 05:08:57.299220 7298 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:38967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
W20260811 05:08:57.299310 7451 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:38967 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.802635 7528 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df65814267fc41b58ce8a3189c7b4d57)
I20260811 05:08:57.802866 7528 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } }
I20260811 05:08:57.803992 7528 leader_election.cc:288] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967), 6ffa08b485524c069089c5a668e5f427 (127.30.88.252:44803)
I20260811 05:08:57.804533 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.804103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59098)
0811 05:08:57.804345 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.804430 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":52,"threads_started":1}
W20260811 05:08:57.804736 6695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.805122 7533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df65814267fc41b58ce8a3189c7b4d57)
I20260811 05:08:57.805404 7533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } }
I20260811 05:08:57.805595 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.805330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59100)
0811 05:08:57.805468 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.805527 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 05:08:57.805750 6695 leader_election.cc:334] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967): Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.806342 7533 leader_election.cc:288] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967), d3f64f2a856d4f23b66e768cf59f84b9 (127.30.88.253:40321)
I20260811 05:08:57.806893 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.806452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59114)
0811 05:08:57.806701 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.806774 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":54,"threads_started":1}
W20260811 05:08:57.807065 6911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.807849 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.807626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59120)
0811 05:08:57.807739 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.807790 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:08:57.808024 6911 leader_election.cc:334] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967): Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.809190 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.804183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:44803 (local address 127.0.0.1:60432)
0811 05:08:57.804385 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.804399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.804504 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.805146 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.805150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.805170 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.805295 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.805303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.806652 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.806655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.807883 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.807897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.808891 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.808893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.808976 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.808995 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.809051 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.809099 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260811 05:08:57.809266 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.804360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.30.88.252:44803)
0811 05:08:57.804633 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.804639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.804657 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:08:57.804705 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.804709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.804760 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.805128 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.805743 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.806519 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.808007 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.808624 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.808774 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.808837 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.809091 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.809166 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
I20260811 05:08:57.809710 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6ffa08b485524c069089c5a668e5f427" is_pre_election: true
I20260811 05:08:57.809896 6944 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f64f2a856d4f23b66e768cf59f84b9 in term 1.
I20260811 05:08:57.810292 6693 leader_election.cc:302] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ffa08b485524c069089c5a668e5f427, d3f64f2a856d4f23b66e768cf59f84b9; no voters: df65814267fc41b58ce8a3189c7b4d57
I20260811 05:08:57.810521 7528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 05:08:57.810585 7528 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader df65814267fc41b58ce8a3189c7b4d57)
I20260811 05:08:57.810624 7528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:08:57.810752 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.806633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.30.88.253:40321)
0811 05:08:57.806921 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.806929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.807038 (+ 109us) server_negotiation.cc:407] Connection header received
0811 05:08:57.807082 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.807085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.807126 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.807207 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.807462 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.808159 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.809647 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.810191 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.810309 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.810364 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.810566 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.810641 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260811 05:08:57.810755 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.806481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:40321 (local address 127.0.0.1:56738)
0811 05:08:57.806761 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.806773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.806875 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.807238 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.807244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.807269 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.807363 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.807369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.808263 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.808269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.809548 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.809558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.810312 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.810316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.810423 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.810446 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.810539 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.810622 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":17,"thread_start_us":53,"threads_started":1}
I20260811 05:08:57.811158 6724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ffa08b485524c069089c5a668e5f427" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" is_pre_election: true
I20260811 05:08:57.811815 7528 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } }
I20260811 05:08:57.812031 6724 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ffa08b485524c069089c5a668e5f427 in current term 2: Already voted for candidate d3f64f2a856d4f23b66e768cf59f84b9 in this term.
I20260811 05:08:57.812160 7528 leader_election.cc:288] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 election: Requested vote from peers df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967), 6ffa08b485524c069089c5a668e5f427 (127.30.88.252:44803)
I20260811 05:08:57.812450 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6ffa08b485524c069089c5a668e5f427"
I20260811 05:08:57.812533 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.812316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59122)
0811 05:08:57.812417 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.812478 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:08:57.812556 6944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:08:57.812556 6910 leader_election.cc:302] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ffa08b485524c069089c5a668e5f427; no voters: d3f64f2a856d4f23b66e768cf59f84b9, df65814267fc41b58ce8a3189c7b4d57
I20260811 05:08:57.813117 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.812924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59134)
0811 05:08:57.813021 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.813067 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 05:08:57.813223 6695 leader_election.cc:334] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967): Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
I20260811 05:08:57.813627 6944 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3f64f2a856d4f23b66e768cf59f84b9 in term 2.
I20260811 05:08:57.813730 7533 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:08:57.813890 6693 leader_election.cc:302] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ffa08b485524c069089c5a668e5f427, d3f64f2a856d4f23b66e768cf59f84b9; no voters: df65814267fc41b58ce8a3189c7b4d57
I20260811 05:08:57.814157 7528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:08:57.814386 7528 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [term 2 LEADER]: Becoming Leader. State: Replica: d3f64f2a856d4f23b66e768cf59f84b9, State: Running, Role: LEADER
I20260811 05:08:57.814621 7528 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } }
I20260811 05:08:57.815778 7539 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f64f2a856d4f23b66e768cf59f84b9. Latest consensus state: current_term: 2 leader_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:57.815898 7539 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:57.816243 7542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:57.818420 7542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:57.818805 7542 catalog_manager.cc:1259] Loaded cluster ID: 5943aa5979b84068ac3ef5c80bc8ff36
I20260811 05:08:57.818897 7542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:57.819456 7542 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:57.819896 7542 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Loaded TSK: 0
I20260811 05:08:57.820556 7542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:08:57.843318 6707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:08:57.846195 6707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:08:57.849144 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.848891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59144)
0811 05:08:57.849015 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.849071 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 05:08:57.849217 6944 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [term 2 FOLLOWER]: Refusing update from remote peer d3f64f2a856d4f23b66e768cf59f84b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 05:08:57.849745 7539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 05:08:57.849910 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.849680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.0.0.1:59148)
0811 05:08:57.849789 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.849839 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:08:57.850018 6695 consensus_peers.cc:487] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 -> Peer df65814267fc41b58ce8a3189c7b4d57 (127.30.88.254:38967): Couldn't send request to peer df65814267fc41b58ce8a3189c7b4d57. Status: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:08:57.851292 7533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f64f2a856d4f23b66e768cf59f84b9. Latest consensus state: current_term: 2 leader_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:57.851411 7533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:57.852447 7539 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:57.852567 7539 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9 [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:57.853091 7533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df65814267fc41b58ce8a3189c7b4d57" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38967 } } peers { permanent_uuid: "d3f64f2a856d4f23b66e768cf59f84b9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 40321 } attrs { promote: false } } peers { permanent_uuid: "6ffa08b485524c069089c5a668e5f427" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 44803 } attrs { promote: false } } }
I20260811 05:08:57.853211 7533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffa08b485524c069089c5a668e5f427 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:08:57.861790 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.857859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39027 (local address 127.0.0.1:44818)
0811 05:08:57.857989 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.858002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.858072 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.858428 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.858431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.858454 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.858538 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.858544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.859432 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.859435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.860561 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.860570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.861401 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.861404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.861490 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.861530 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.861610 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.861684 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 05:08:57.862020 7550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.857977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.30.88.195:39027)
0811 05:08:57.858182 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.858185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.858197 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:08:57.858279 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.858282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.858330 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.858423 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.858654 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.859332 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.860683 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.861279 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.861405 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.861484 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.861649 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.861728 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260811 05:08:57.862661 7374 tablet_service.cc:1467] Processing CreateTablet for tablet 9307cec034884124942a1db426707501 (DEFAULT_TABLE table=test-table [id=a64cef4d9687470385150d59815b1df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:57.862906 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.858580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46243 (local address 127.0.0.1:52134)
0811 05:08:57.858804 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.858816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.859243 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.859526 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.859529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.859545 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.859623 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.859628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.860329 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.860332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.861420 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.861430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.862518 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.862522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.862612 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.862633 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.862703 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.862771 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260811 05:08:57.862998 7374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9307cec034884124942a1db426707501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:57.863308 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.859059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.30.88.193:46243)
0811 05:08:57.859303 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.859308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.859322 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:08:57.859372 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.859377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.859429 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.859512 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.859735 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.860216 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.861559 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.862152 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.862276 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.862337 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.862914 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.863180 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260811 05:08:57.863471 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.858415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:46527 (local address 127.0.0.1:34512)
0811 05:08:57.858775 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.858787 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.858899 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.859649 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.859652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.859667 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.859743 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.859748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.860748 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.860752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.862338 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.862348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.863128 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.863131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.863205 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.863226 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.863271 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.863333 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":685}
I20260811 05:08:57.863766 7061 tablet_service.cc:1467] Processing CreateTablet for tablet 9307cec034884124942a1db426707501 (DEFAULT_TABLE table=test-table [id=a64cef4d9687470385150d59815b1df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:57.864002 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.858542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34512 (local address 127.30.88.194:46527)
0811 05:08:57.859414 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.859419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.859432 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:57.859478 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.859481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.859533 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.859838 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.859950 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.860649 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.862460 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.862986 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.863716 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.863769 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.863805 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.863876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":91,"threads_started":1}
I20260811 05:08:57.864032 7061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9307cec034884124942a1db426707501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:57.864481 7219 tablet_service.cc:1467] Processing CreateTablet for tablet 9307cec034884124942a1db426707501 (DEFAULT_TABLE table=test-table [id=a64cef4d9687470385150d59815b1df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:08:57.864744 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9307cec034884124942a1db426707501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:57.865625 7555 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Bootstrap starting.
I20260811 05:08:57.866375 7556 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: Bootstrap starting.
I20260811 05:08:57.866626 7555 tablet_bootstrap.cc:654] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:57.867280 7556 tablet_bootstrap.cc:654] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:57.867455 7557 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Bootstrap starting.
I20260811 05:08:57.868283 7555 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: No bootstrap required, opened a new log
I20260811 05:08:57.868336 7557 tablet_bootstrap.cc:654] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:57.868321 7556 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: No bootstrap required, opened a new log
I20260811 05:08:57.868379 7555 ts_tablet_manager.cc:1397] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:08:57.868422 7556 ts_tablet_manager.cc:1397] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:57.868839 7555 raft_consensus.cc:348] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.868803 7556 raft_consensus.cc:348] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.868928 7555 raft_consensus.cc:374] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:57.868933 7556 raft_consensus.cc:374] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:57.868963 7555 raft_consensus.cc:729] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bd0b78ec53475980c1f8b9752250a4, State: Initialized, Role: FOLLOWER
I20260811 05:08:57.868971 7556 raft_consensus.cc:729] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f06f5b075943c29216783a19ae558a, State: Initialized, Role: FOLLOWER
I20260811 05:08:57.869087 7555 consensus_queue.cc:260] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.869097 7556 consensus_queue.cc:260] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.869423 7556 ts_tablet_manager.cc:1428] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:57.869444 7555 ts_tablet_manager.cc:1428] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:57.869760 7557 tablet_bootstrap.cc:492] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: No bootstrap required, opened a new log
I20260811 05:08:57.869841 7557 ts_tablet_manager.cc:1397] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:57.870208 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.869822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:51285)
0811 05:08:57.870057 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.870127 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
I20260811 05:08:57.870231 7557 raft_consensus.cc:348] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.870326 7557 raft_consensus.cc:374] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:57.870358 7557 raft_consensus.cc:729] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912d16ff05c64cb49f23321f33dc971a, State: Initialized, Role: FOLLOWER
I20260811 05:08:57.870486 7557 consensus_queue.cc:260] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.870805 7557 ts_tablet_manager.cc:1428] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:08:57.872354 7453 heartbeater.cc:503] Master 127.30.88.253:40321 was elected leader, sending a full tablet report...
I20260811 05:08:57.873220 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.871873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:49645)
0811 05:08:57.873051 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.873105 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1090,"thread_start_us":86,"threads_started":1}
I20260811 05:08:57.873607 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.871418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:53101)
0811 05:08:57.873415 (+ 1997us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.873467 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1940,"thread_start_us":87,"threads_started":1}
I20260811 05:08:57.873977 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.873758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:47525)
0811 05:08:57.873876 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.873924 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 05:08:57.874229 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.873264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:53851)
0811 05:08:57.874117 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.874165 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":727}
I20260811 05:08:57.874927 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.874498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:34753)
0811 05:08:57.874785 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.874827 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260811 05:08:57.874955 7143 heartbeater.cc:503] Master 127.30.88.253:40321 was elected leader, sending a full tablet report...
I20260811 05:08:57.951071 7564 raft_consensus.cc:482] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:08:57.951197 7564 raft_consensus.cc:504] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.952409 7564 leader_election.cc:288] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527), 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:57.956506 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.952800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:39165 (local address 127.30.88.194:46527)
0811 05:08:57.952972 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.952977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.953020 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:08:57.953117 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.953122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.953189 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.953275 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.953548 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.954056 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.955509 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.955910 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.956057 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.956113 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.956310 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.956384 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 05:08:57.956535 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.952673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:46527 (local address 127.30.88.195:39165)
0811 05:08:57.952829 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.952862 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.953013 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.953284 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.953287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.953304 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.953429 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.953435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.954166 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.954170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.955384 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.955394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.956053 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.956057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.956157 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.956180 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.956295 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.956421 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":63}
I20260811 05:08:57.956715 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.953121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:34273 (local address 127.30.88.193:46243)
0811 05:08:57.953271 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.953277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.953338 (+ 61us) server_negotiation.cc:407] Connection header received
0811 05:08:57.953465 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.953468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.953514 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.953597 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:08:57.953859 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.954403 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.955687 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.956095 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.956250 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.956295 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.956510 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.956578 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 05:08:57.956876 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.952978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46243 (local address 127.30.88.195:34273)
0811 05:08:57.953268 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.953280 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:57.953372 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:57.953612 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:57.953615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:57.953630 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:08:57.953738 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.953751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.954529 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.954534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.955555 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:57.955565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.956249 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.956252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:57.956378 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.956411 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:08:57.956492 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:08:57.956678 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":41,"thread_start_us":127,"threads_started":1}
I20260811 05:08:57.957002 7243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9307cec034884124942a1db426707501" candidate_uuid: "88bd0b78ec53475980c1f8b9752250a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "912d16ff05c64cb49f23321f33dc971a" is_pre_election: true
I20260811 05:08:57.957244 7243 raft_consensus.cc:2454] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88bd0b78ec53475980c1f8b9752250a4 in term 0.
I20260811 05:08:57.957311 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9307cec034884124942a1db426707501" candidate_uuid: "88bd0b78ec53475980c1f8b9752250a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f06f5b075943c29216783a19ae558a" is_pre_election: true
I20260811 05:08:57.957434 7087 raft_consensus.cc:2454] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88bd0b78ec53475980c1f8b9752250a4 in term 0.
I20260811 05:08:57.957656 7327 leader_election.cc:302] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88bd0b78ec53475980c1f8b9752250a4, 912d16ff05c64cb49f23321f33dc971a; no voters:
I20260811 05:08:57.957872 7564 raft_consensus.cc:2781] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:08:57.957926 7564 raft_consensus.cc:482] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:08:57.957958 7564 raft_consensus.cc:3037] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:57.959018 7564 raft_consensus.cc:504] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.959326 7564 leader_election.cc:288] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [CANDIDATE]: Term 1 election: Requested vote from peers 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527), 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243)
I20260811 05:08:57.959669 7243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9307cec034884124942a1db426707501" candidate_uuid: "88bd0b78ec53475980c1f8b9752250a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "912d16ff05c64cb49f23321f33dc971a"
I20260811 05:08:57.959791 7243 raft_consensus.cc:3037] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:57.959812 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9307cec034884124942a1db426707501" candidate_uuid: "88bd0b78ec53475980c1f8b9752250a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f06f5b075943c29216783a19ae558a"
I20260811 05:08:57.959926 7087 raft_consensus.cc:3037] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:57.960711 7243 raft_consensus.cc:2454] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88bd0b78ec53475980c1f8b9752250a4 in term 1.
I20260811 05:08:57.960768 7087 raft_consensus.cc:2454] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88bd0b78ec53475980c1f8b9752250a4 in term 1.
I20260811 05:08:57.961052 7327 leader_election.cc:302] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f06f5b075943c29216783a19ae558a, 88bd0b78ec53475980c1f8b9752250a4; no voters:
I20260811 05:08:57.961200 7564 raft_consensus.cc:2781] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:57.961397 7564 raft_consensus.cc:686] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 1 LEADER]: Becoming Leader. State: Replica: 88bd0b78ec53475980c1f8b9752250a4, State: Running, Role: LEADER
I20260811 05:08:57.961643 7564 consensus_queue.cc:237] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } }
I20260811 05:08:57.961987 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.961804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:57795)
0811 05:08:57.961890 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.961940 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 05:08:57.963967 6707 catalog_manager.cc:5495] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "88bd0b78ec53475980c1f8b9752250a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88bd0b78ec53475980c1f8b9752250a4" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 } health_report { overall_health: UNKNOWN } } }
I20260811 05:08:57.986253 7550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.983150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.30.88.195:39027)
0811 05:08:57.983251 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:57.983255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:57.983268 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:08:57.983450 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:57.983452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:57.983501 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:57.983581 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:08:57.983775 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.984218 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.984947 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:57.985176 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:57.985249 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:57.985470 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:08:57.985871 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:08:57.985956 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:57.986023 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:08:57.986069 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:08:57.988620 7087 raft_consensus.cc:1261] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a [term 1 FOLLOWER]: Refusing update from remote peer 88bd0b78ec53475980c1f8b9752250a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:57.988878 7243 raft_consensus.cc:1261] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Refusing update from remote peer 88bd0b78ec53475980c1f8b9752250a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:08:57.989092 7571 consensus_queue.cc:1035] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f06f5b075943c29216783a19ae558a" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:57.989459 7564 consensus_queue.cc:1035] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "912d16ff05c64cb49f23321f33dc971a" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:08:57.990911 7299 heartbeater.cc:503] Master 127.30.88.253:40321 was elected leader, sending a full tablet report...
I20260811 05:08:57.991467 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.990880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:38047)
0811 05:08:57.991292 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.991368 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
I20260811 05:08:57.991514 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.991080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:39007)
0811 05:08:57.991377 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.991430 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 05:08:57.994037 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.993635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.194:60197)
0811 05:08:57.993926 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.993975 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260811 05:08:57.994164 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.993849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:58987)
0811 05:08:57.994038 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.994097 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 05:08:57.997334 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.997086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.195:50545)
0811 05:08:57.997238 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.997280 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 05:08:57.998092 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:57.997906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38967 (local address 127.30.88.193:60395)
0811 05:08:57.997983 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:57.998031 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 05:08:58.000346 6710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:58.000507 6710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:08:58.004112 6710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3f64f2a856d4f23b66e768cf59f84b9: Sending DeleteTablet for 3 replicas of tablet 9307cec034884124942a1db426707501
I20260811 05:08:58.004776 7374 tablet_service.cc:1514] Processing DeleteTablet for tablet 9307cec034884124942a1db426707501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:57 UTC) from {username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:44818
I20260811 05:08:58.004892 7219 tablet_service.cc:1514] Processing DeleteTablet for tablet 9307cec034884124942a1db426707501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:57 UTC) from {username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:34512
I20260811 05:08:58.005190 7591 tablet_replica.cc:330] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: stopping tablet replica
I20260811 05:08:58.005276 7590 tablet_replica.cc:330] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: stopping tablet replica
I20260811 05:08:58.005326 7591 raft_consensus.cc:2229] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:08:58.005371 7590 raft_consensus.cc:2229] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:08:58.005373 7061 tablet_service.cc:1514] Processing DeleteTablet for tablet 9307cec034884124942a1db426707501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:08:57 UTC) from {username='oryx', principal='oryx/127.30.88.253@KRBTEST.COM'} at 127.0.0.1:52134
I20260811 05:08:58.005453 7591 raft_consensus.cc:2258] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:58.005702 7590 raft_consensus.cc:2258] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:08:58.005964 7592 tablet_replica.cc:330] T 9307cec034884124942a1db426707501 P 58f06f5b075943c29216783a19ae558a: stopping tablet replica
I20260811 05:08:58.006055 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6881
I20260811 05:08:58.006178 7591 ts_tablet_manager.cc:1905] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:58.013341 7590 ts_tablet_manager.cc:1905] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:08:58.014073 7591 ts_tablet_manager.cc:1918] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:58.014174 7591 log.cc:1196] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/9307cec034884124942a1db426707501
I20260811 05:08:58.014497 7591 ts_tablet_manager.cc:1939] T 9307cec034884124942a1db426707501 P 912d16ff05c64cb49f23321f33dc971a: Deleting consensus metadata
I20260811 05:08:58.015316 6695 catalog_manager.cc:4841] TS 912d16ff05c64cb49f23321f33dc971a (127.30.88.194:46527): tablet 9307cec034884124942a1db426707501 (table test-table [id=a64cef4d9687470385150d59815b1df0]) successfully deleted
I20260811 05:08:58.016309 7590 ts_tablet_manager.cc:1918] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:08:58.016405 7590 log.cc:1196] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/9307cec034884124942a1db426707501
I20260811 05:08:58.016744 7590 ts_tablet_manager.cc:1939] T 9307cec034884124942a1db426707501 P 88bd0b78ec53475980c1f8b9752250a4: Deleting consensus metadata
I20260811 05:08:58.017400 6693 catalog_manager.cc:4841] TS 88bd0b78ec53475980c1f8b9752250a4 (127.30.88.195:39027): tablet 9307cec034884124942a1db426707501 (table test-table [id=a64cef4d9687470385150d59815b1df0]) successfully deleted
I20260811 05:08:58.020311 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 7149
I20260811 05:08:58.020346 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:58.020080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46243 (local address 127.0.0.1:52146)
0811 05:08:58.020204 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:58.020279 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:46243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:08:58.023411 6695 catalog_manager.cc:4584] TS 58f06f5b075943c29216783a19ae558a (127.30.88.193:46243): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9307cec034884124942a1db426707501: Network error: Client connection negotiation failed: client connection to 127.30.88.193:46243: connect: Connection refused (error 111)
I20260811 05:08:58.031822 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 7304
I20260811 05:08:58.040370 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6555
I20260811 05:08:58.047930 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 6671
2026-08-11T05:08:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5472 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:08:58 dist-test-slave-7c35 krb5kdc[7595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:08:58 dist-test-slave-7c35 krb5kdc[7595](info): set up 2 sockets
Aug 11 05:08:58 dist-test-slave-7c35 krb5kdc[7595](info): commencing operation
krb5kdc: starting...
W20260811 05:08:59.166963 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:08:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab.
Entry for principal oryx/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab.
I20260811 05:08:59.290259 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34311
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab
--principal=oryx/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:34311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:59.397713 7610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:59.397953 7610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:59.397990 7610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:59.401899 7610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:08:59.401964 7610 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:08:59.401983 7610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:59.401999 7610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:59.402021 7610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:08:59.402040 7610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:08:59.407171 7610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:34311
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.254.keytab
--principal=oryx/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:59.408113 7610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:59.409039 7610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:59.417393 7617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.417460 7616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.417505 7619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:59.417605 7610 server_base.cc:1034] running on GCE node
I20260811 05:08:59.418218 7610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:59.419303 7610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:59.420456 7610 hybrid_clock.cc:648] HybridClock initialized: now 1786424939420437 us; error 28 us; skew 500 ppm
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:59.423069 7610 init.cc:377] Logged in from keytab as oryx/127.30.88.254@KRBTEST.COM (short username oryx)
I20260811 05:08:59.423831 7610 webserver.cc:460] Webserver started at http://127.30.88.254:41991/ using document root <none> and password file <none>
I20260811 05:08:59.424345 7610 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:59.424428 7610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:59.424645 7610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:59.426578 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "a54c94c2361847c88e8cbb49f5a0476b"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "51d1cc1292f5a87f9b5cb90e85e7be45"
server_key_iv: "24ba4c38b5be9653eee7d242952b50c3"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.427117 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "a54c94c2361847c88e8cbb49f5a0476b"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "51d1cc1292f5a87f9b5cb90e85e7be45"
server_key_iv: "24ba4c38b5be9653eee7d242952b50c3"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.430631 7610 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:08:59.432906 7626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:59.434064 7610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:08:59.434194 7610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "a54c94c2361847c88e8cbb49f5a0476b"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "51d1cc1292f5a87f9b5cb90e85e7be45"
server_key_iv: "24ba4c38b5be9653eee7d242952b50c3"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.434293 7610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:59.458137 7610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:59.460986 7610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:59.461177 7610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:59.468781 7610 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:34311
I20260811 05:08:59.468791 7696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:34311 every 8 connection(s)
I20260811 05:08:59.469731 7610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:08:59.472496 7697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:08:59.475883 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 7610
I20260811 05:08:59.476037 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:08:59.476315 31075 external_mini_cluster.cc:1428] Setting key 7bfbe638b8df8255b1769324afcd946f
I20260811 05:08:59.478257 7697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Bootstrap starting.
I20260811 05:08:59.480376 7697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Neither blocks nor log segments found. Creating new log.
I20260811 05:08:59.481082 7697 log.cc:824] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:08:59.482784 7697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: No bootstrap required, opened a new log
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:59.486945 7697 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } }
I20260811 05:08:59.487151 7697 raft_consensus.cc:374] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:08:59.487231 7697 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54c94c2361847c88e8cbb49f5a0476b, State: Initialized, Role: FOLLOWER
I20260811 05:08:59.487664 7697 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } }
I20260811 05:08:59.487777 7697 raft_consensus.cc:388] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:08:59.487849 7697 raft_consensus.cc:482] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:08:59.487941 7697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:08:59.488745 7697 raft_consensus.cc:504] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } }
I20260811 05:08:59.489131 7697 leader_election.cc:302] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a54c94c2361847c88e8cbb49f5a0476b; no voters:
I20260811 05:08:59.489516 7697 leader_election.cc:288] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:08:59.489581 7702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:08:59.489761 7702 raft_consensus.cc:686] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [term 1 LEADER]: Becoming Leader. State: Replica: a54c94c2361847c88e8cbb49f5a0476b, State: Running, Role: LEADER
I20260811 05:08:59.490053 7702 consensus_queue.cc:237] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } }
I20260811 05:08:59.490264 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.477617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.30.88.254:34311)
0811 05:08:59.478040 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:59.478055 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:59.478088 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:08:59.478725 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:59.478742 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:59.479080 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:59.479371 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:59.480165 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.481221 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.482077 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.482530 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.482691 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.485450 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:59.485455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:59.485467 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:59.485492 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:59.487235 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.487658 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.487663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.487669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.487807 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.488117 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.488120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.488122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.488494 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:59.488701 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:59.488967 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 05:08:59.489299 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260811 05:08:59.490756 7697 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:08:59.491696 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a54c94c2361847c88e8cbb49f5a0476b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } } }
I20260811 05:08:59.491845 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:59.492174 7704 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a54c94c2361847c88e8cbb49f5a0476b. Latest consensus state: current_term: 1 leader_uuid: "a54c94c2361847c88e8cbb49f5a0476b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c94c2361847c88e8cbb49f5a0476b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34311 } } }
I20260811 05:08:59.492264 7704 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b [sys.catalog]: This master's current role is: LEADER
I20260811 05:08:59.492974 7716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:08:59.495769 7716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:08:59.501361 7716 catalog_manager.cc:1347] Generated new cluster ID: a14b020f0e674c35bc5031c0de7b5163
I20260811 05:08:59.501422 7716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:08:59.532208 7716 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:08:59.533087 7716 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:08:59.548666 7716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Generated new TSK 0
I20260811 05:08:59.549324 7716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
I20260811 05:08:59.579900 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34311
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:59.712843 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:59.713173 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:59.713233 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:59.717177 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:59.717224 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:59.717260 7729 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:59.717290 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:08:59.722050 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:34311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:59.722903 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:59.723778 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:59.732815 7739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.732805 7735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.732821 7736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:59.733063 7729 server_base.cc:1034] running on GCE node
I20260811 05:08:59.733559 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:59.734143 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:59.735332 7729 hybrid_clock.cc:648] HybridClock initialized: now 1786424939735308 us; error 43 us; skew 500 ppm
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:59.738143 7729 init.cc:377] Logged in from keytab as oryx/127.30.88.193@KRBTEST.COM (short username oryx)
I20260811 05:08:59.739060 7729 webserver.cc:460] Webserver started at http://127.30.88.193:44383/ using document root <none> and password file <none>
I20260811 05:08:59.739593 7729 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:59.739676 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:59.739888 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:59.741809 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "7305752aebfa457ea6029a0e18656837"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "d0a8fef0814094fa61360ffc7bc50c60"
server_key_iv: "8d87addd0962ff635d4b37b0d4718d81"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.742333 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "7305752aebfa457ea6029a0e18656837"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "d0a8fef0814094fa61360ffc7bc50c60"
server_key_iv: "8d87addd0962ff635d4b37b0d4718d81"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.745888 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:59.748265 7750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:59.749382 7729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:59.749516 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "7305752aebfa457ea6029a0e18656837"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "d0a8fef0814094fa61360ffc7bc50c60"
server_key_iv: "8d87addd0962ff635d4b37b0d4718d81"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.749617 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:59.764820 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:59.767540 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:59.767737 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:59.769045 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:59.769132 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:59.769202 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:59.769248 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:59.780252 7729 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:34953
I20260811 05:08:59.780326 7877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:34953 every 8 connection(s)
I20260811 05:08:59.781070 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:08:59.783263 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.769833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:60731 (local address 127.30.88.254:34311)
0811 05:08:59.769961 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:59.769964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:59.771016 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 05:08:59.771940 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:59.771944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:59.772005 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:59.772108 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:59.773922 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.774601 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.776506 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.776812 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.776963 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.779654 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:59.779658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:59.779661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:59.779692 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:59.781184 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.781757 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.781761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.781763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.781853 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.782211 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.782215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.782216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.782395 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:59.782502 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:59.782976 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 05:08:59.783098 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:08:59.783910 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.770372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.193:60731)
0811 05:08:59.770827 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:59.770914 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:59.771744 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:59.772432 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:59.772442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:59.773018 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:59.773682 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.773706 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.774752 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.774758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.776350 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.776367 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.777000 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.777004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.777212 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.777740 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:59.777755 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:59.779478 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:59.781301 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.781307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.781320 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.781636 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.781953 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.781959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.781964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.782098 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.782501 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:59.782507 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:59.782726 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 05:08:59.782908 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 05:08:59.783194 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260811 05:08:59.785073 7878 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34311
I20260811 05:08:59.785228 7878 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:59.785753 7878 heartbeater.cc:511] Master 127.30.88.254:34311 requested a full tablet report, sending...
I20260811 05:08:59.786553 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 7729
I20260811 05:08:59.786681 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:08:59.787029 31075 external_mini_cluster.cc:1428] Setting key fa82d4daab6abed04b1c25d651ef264a
I20260811 05:08:59.787446 7644 ts_manager.cc:194] Registered new tserver with Master: 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953)
I20260811 05:08:59.788793 7644 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:60731
I20260811 05:08:59.795033 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.787122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.193:40711)
0811 05:08:59.787589 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:59.787603 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:59.787714 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:59.787942 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:59.787945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:59.788168 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:59.788312 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.788317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.789065 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.789068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.789798 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.789808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.790290 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.790295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.790394 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.790948 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:59.790968 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:59.791487 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:59.793500 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.793503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.793504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.793688 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.794006 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.794008 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.794009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.794119 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.794595 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:59.794597 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:59.794670 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 05:08:59.794747 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:08:59.794841 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":102,"threads_started":1}
I20260811 05:08:59.795243 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.787183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:40711 (local address 127.30.88.254:34311)
0811 05:08:59.787284 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:59.787288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:59.787676 (+ 388us) server_negotiation.cc:407] Connection header received
0811 05:08:59.787789 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:59.787792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:59.787840 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:59.787922 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:59.788415 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.788917 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.789911 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.790138 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.790239 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.791647 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:59.791652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:59.791656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:59.791685 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:59.793369 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.793789 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.793795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.793800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.793903 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.794225 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.794232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.794236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.794490 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:59.794594 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:59.794841 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 05:08:59.795000 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for oryx/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
I20260811 05:08:59.815920 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34311
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:08:59.923929 7887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:08:59.924160 7887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:08:59.924229 7887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:08:59.928043 7887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:08:59.928145 7887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:08:59.928241 7887 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:08:59.928311 7887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:08:59.933362 7887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:34311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:08:59.934248 7887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:08:59.935292 7887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:08:59.944095 7895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.944108 7898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:08:59.944099 7896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:08:59.944478 7887 server_base.cc:1034] running on GCE node
I20260811 05:08:59.944921 7887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:08:59.945484 7887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:08:59.946679 7887 hybrid_clock.cc:648] HybridClock initialized: now 1786424939946662 us; error 31 us; skew 500 ppm
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:08:59.949468 7887 init.cc:377] Logged in from keytab as oryx/127.30.88.194@KRBTEST.COM (short username oryx)
I20260811 05:08:59.950335 7887 webserver.cc:460] Webserver started at http://127.30.88.194:46059/ using document root <none> and password file <none>
I20260811 05:08:59.950920 7887 fs_manager.cc:359] Metadata directory not provided
I20260811 05:08:59.951016 7887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:08:59.951256 7887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:08:59.953322 7887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "efd147a500d8419ca0232fd31536c088"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "addae0d8a0a1df42c29c7d0a48c43f59"
server_key_iv: "e2420d27ec19e3578a658680fad8c8d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.953945 7887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "efd147a500d8419ca0232fd31536c088"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "addae0d8a0a1df42c29c7d0a48c43f59"
server_key_iv: "e2420d27ec19e3578a658680fad8c8d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.957657 7887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:08:59.960031 7906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:59.961123 7887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:08:59.961257 7887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "efd147a500d8419ca0232fd31536c088"
format_stamp: "Formatted at 2026-08-11 05:08:59 on dist-test-slave-7c35"
server_key: "addae0d8a0a1df42c29c7d0a48c43f59"
server_key_iv: "e2420d27ec19e3578a658680fad8c8d2"
server_key_version: "encryptionkey@0"
I20260811 05:08:59.961388 7887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:08:59.971659 7887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:08:59.974105 7887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:08:59.974291 7887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:08:59.975739 7887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:08:59.975827 7887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:08:59.975885 7887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:08:59.975922 7887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:08:59 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:08:59.988108 7887 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:34501
I20260811 05:08:59.988914 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.976290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:48549 (local address 127.30.88.254:34311)
0811 05:08:59.976408 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:59.976412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:59.977376 (+ 964us) server_negotiation.cc:407] Connection header received
0811 05:08:59.978232 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:59.978235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:59.978295 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:59.978371 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:59.979592 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.980122 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.981393 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.981674 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.981780 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.984828 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:59.984832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:59.984835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:59.984866 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:59.986689 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.987190 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.987194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.987196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.987326 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:08:59.987657 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:08:59.987661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:08:59.987663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:08:59.987938 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:08:59.988076 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:08:59.988570 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 05:08:59.988701 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:08:59.989121 7887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:08:59.989517 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.976756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.194:48549)
0811 05:08:59.977243 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:59.977282 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:59.978037 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:59.978550 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:59.978560 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:59.978984 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:59.979459 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.979473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.980243 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.980247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.981271 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.981281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.981812 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.981815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.981972 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.982609 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:59.982628 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:59.984667 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:59.986807 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.986814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.986829 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.987090 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.987421 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.987423 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.987424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:08:59.987548 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:08:59.988065 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:08:59.988070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:08:59.988313 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 05:08:59.988477 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 05:08:59.988686 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260811 05:08:59.989950 8043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:34501 every 8 connection(s)
I20260811 05:08:59.991070 8044 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34311
I20260811 05:08:59.991257 8044 heartbeater.cc:464] Registering TS with master...
I20260811 05:08:59.991438 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 7887
I20260811 05:08:59.991582 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:08:59.991758 8044 heartbeater.cc:511] Master 127.30.88.254:34311 requested a full tablet report, sending...
I20260811 05:08:59.991925 31075 external_mini_cluster.cc:1428] Setting key 87f0caf28a8bf568e8b6572062ee1573
I20260811 05:08:59.992854 7645 ts_manager.cc:194] Registered new tserver with Master: efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501)
I20260811 05:08:59.993530 7645 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:48549
I20260811 05:09:00.001523 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.993704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:46883 (local address 127.30.88.254:34311)
0811 05:08:59.994965 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0811 05:08:59.994968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:08:59.994977 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:08:59.995003 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:08:59.995004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:08:59.995033 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:08:59.995112 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:08:59.995563 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.996298 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.997108 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.997383 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.997489 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.998460 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:08:59.998463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:08:59.998466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:08:59.998488 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:08:59.999854 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.000319 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.000322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.000324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.000410 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.000677 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.000681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.000683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.000885 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.001001 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.001250 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.001356 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192}
I20260811 05:09:00.001531 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:08:59.993638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.194:46883)
0811 05:08:59.993904 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 05:08:59.993917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:08:59.994003 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:08:59.995128 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:08:59.995131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:08:59.995357 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:08:59.995452 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.995458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.996451 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.996453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.996993 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:08:59.997002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:08:59.997496 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:08:59.997499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:08:59.997601 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:08:59.998055 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:08:59.998065 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:08:59.998332 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:08:59.999982 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:08:59.999985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:08:59.999987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.000210 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.000500 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.000504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.000506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.000588 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.000990 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:00.000994 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:00.001089 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.001193 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.001324 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
I20260811 05:09:00.020094 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34311
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:00.130813 8053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:00.131101 8053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:00.131187 8053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:00.134820 8053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:00.134924 8053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:00.134965 8053 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:09:00.135032 8053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:00.139842 8053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.88.254:34311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:00.140714 8053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:00.141703 8053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:00.150748 8059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:00.150882 8060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:00.150735 8062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:00.151017 8053 server_base.cc:1034] running on GCE node
I20260811 05:09:00.151582 8053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:00.152154 8053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:00.153350 8053 hybrid_clock.cc:648] HybridClock initialized: now 1786424940153306 us; error 41 us; skew 500 ppm
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:00.160591 8053 init.cc:377] Logged in from keytab as oryx/127.30.88.195@KRBTEST.COM (short username oryx)
I20260811 05:09:00.161497 8053 webserver.cc:460] Webserver started at http://127.30.88.195:33941/ using document root <none> and password file <none>
I20260811 05:09:00.162046 8053 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:00.162133 8053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:00.162369 8053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:00.164500 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "42577ab07f53476f94605c701146acbe"
format_stamp: "Formatted at 2026-08-11 05:09:00 on dist-test-slave-7c35"
server_key: "c20b894e555a2a8060c4ecd279fdf6a3"
server_key_iv: "7ba9c2c9ef23ae6672e8ff8f553b8e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:00.165088 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "42577ab07f53476f94605c701146acbe"
format_stamp: "Formatted at 2026-08-11 05:09:00 on dist-test-slave-7c35"
server_key: "c20b894e555a2a8060c4ecd279fdf6a3"
server_key_iv: "7ba9c2c9ef23ae6672e8ff8f553b8e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:00.168939 8053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:09:00.171341 8073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:00.172472 8053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:09:00.172614 8053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "42577ab07f53476f94605c701146acbe"
format_stamp: "Formatted at 2026-08-11 05:09:00 on dist-test-slave-7c35"
server_key: "c20b894e555a2a8060c4ecd279fdf6a3"
server_key_iv: "7ba9c2c9ef23ae6672e8ff8f553b8e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:00.172724 8053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:00.190306 8053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:00.193774 8053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:00.193971 8053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:00.195478 8053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:00.195547 8053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:00.195616 8053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:00.195662 8053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:09:00.206594 8053 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:34449
I20260811 05:09:00.206622 8205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:34449 every 8 connection(s)
I20260811 05:09:00.207631 8053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:00.209609 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.196079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:40847 (local address 127.30.88.254:34311)
0811 05:09:00.196226 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.196231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.197075 (+ 844us) server_negotiation.cc:407] Connection header received
0811 05:09:00.198002 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.198007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.198065 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.198177 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.199525 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.200453 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.201673 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.201909 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.202023 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.205752 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:00.205755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:00.205757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:00.205782 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:00.207382 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.207973 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.207976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.207977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.208058 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.208639 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.208642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.208643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.208854 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.208936 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.209379 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.209490 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:09:00.210379 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.196481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.195:40847)
0811 05:09:00.196906 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.196963 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.197797 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.198333 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.198342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.198795 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:00.199317 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.199330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.200597 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.200601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.201541 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.201552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.202058 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.202063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.202274 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.202926 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:00.202960 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:00.205587 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:00.207492 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.207498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.207510 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.207859 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.208204 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.208207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.208209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.208362 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.208960 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:00.208964 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:00.209169 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.209324 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.209571 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":121,"threads_started":1}
I20260811 05:09:00.211537 8207 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34311
I20260811 05:09:00.211688 8207 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:00.212160 8207 heartbeater.cc:511] Master 127.30.88.254:34311 requested a full tablet report, sending...
I20260811 05:09:00.213166 7644 ts_manager.cc:194] Registered new tserver with Master: 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.213912 7644 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:40847
I20260811 05:09:00.216758 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8053
I20260811 05:09:00.216908 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:09:00.217206 31075 external_mini_cluster.cc:1428] Setting key e821a3647f7000aa4aeec6f853d7dc89
I20260811 05:09:00.219347 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:09:00.220644 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.213676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59065 (local address 127.30.88.254:34311)
0811 05:09:00.213806 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.213810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.214063 (+ 253us) server_negotiation.cc:407] Connection header received
0811 05:09:00.214260 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.214263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.214313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.214391 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.214830 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.215375 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.216146 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.216351 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.216436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.217460 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:00.217464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:00.217469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:00.217499 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:00.219086 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.219540 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.219543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.219544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.219622 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.219910 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.219912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.219913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.220067 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.220139 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.220454 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.220541 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:09:00.220842 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.213606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.30.88.195:59065)
0811 05:09:00.213984 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.213998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.214125 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.214415 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.214417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.214569 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:00.214676 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.214681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.215509 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.215511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.216045 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.216052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.216451 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.216453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.216529 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.216989 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:00.217010 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:00.217333 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:00.219177 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.219182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.219184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.219454 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.219710 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.219713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.219715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.219805 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.220157 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:00.220162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:00.220275 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.220389 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.220539 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:09:00.243994 7700 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:00.238412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.30.88.254:34311)
0811 05:09:00.238532 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.238536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.238548 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:09:00.238582 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.238584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.238617 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.238683 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.239410 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.239905 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.240701 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.240911 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.240986 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.243828 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42722: BlockingRecv error: recv got EOF from 127.0.0.1:42722 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:09:00.252693 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.245123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.30.88.254:34311)
0811 05:09:00.245258 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.245263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.245405 (+ 142us) server_negotiation.cc:407] Connection header received
0811 05:09:00.245521 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.245525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.245566 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.245646 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.246207 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.246685 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.247529 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.247760 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.247863 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.249939 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:00.249942 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:00.249944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:00.249966 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:00.251276 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.251647 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.251649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.251650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.251720 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.251964 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.251967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.251968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.252142 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.252243 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.252429 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.252511 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:09:00.255256 7644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:09:00.258157 7644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:00.272120 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.266281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34953 (local address 127.0.0.1:57784)
0811 05:09:00.266793 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.266845 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.266996 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.268036 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.268043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.268065 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.268233 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.268240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.269455 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.269460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.270619 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.270629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.271601 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.271605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.271720 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.271759 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.271869 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.271982 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260811 05:09:00.272179 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.266434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.30.88.193:34953)
0811 05:09:00.267429 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.267460 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.267486 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:09:00.267566 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.267571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.267835 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.268393 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.268524 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.269349 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.270742 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.271485 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.271588 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.271736 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.271932 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.272036 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":82,"threads_started":1}
I20260811 05:09:00.273001 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.267843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.30.88.194:34501)
0811 05:09:00.268268 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.268279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.268300 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:09:00.268376 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.268381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.268567 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.268718 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.268935 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.269927 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.271358 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.272088 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.272192 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.272376 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.272767 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.272854 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":125,"threads_started":1}
I20260811 05:09:00.273067 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.266466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:34449 (local address 127.0.0.1:35670)
0811 05:09:00.267097 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.267109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.267192 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.268188 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.268191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.268209 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.268290 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.268295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.269788 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.269791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.271075 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.271087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.272107 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.272111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.272494 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.272570 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.272642 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.272946 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":44,"thread_start_us":92,"threads_started":1}
I20260811 05:09:00.273353 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.267544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34501 (local address 127.0.0.1:44714)
0811 05:09:00.267911 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.267956 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.268036 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.268713 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.268716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.268734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.268822 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.268827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.270059 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.270062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.271236 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.271243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.272445 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.272448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.272591 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.272615 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.272690 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.273233 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":504,"thread_start_us":80,"threads_started":1}
I20260811 05:09:00.273042 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.267322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.30.88.195:34449)
0811 05:09:00.267617 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.267630 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.267671 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:09:00.267748 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.267753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.267996 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.268208 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.268378 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.269664 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.271207 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.271972 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.272563 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.272735 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.272788 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.272890 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":118,"threads_started":1}
I20260811 05:09:00.275362 7804 tablet_service.cc:1467] Processing CreateTablet for tablet 29508b468330412fa2d9f77a31b4789d (DEFAULT_TABLE table=test-table [id=fb020eb59e91443b8cdcee9614dbeece]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.275878 7967 tablet_service.cc:1467] Processing CreateTablet for tablet 29508b468330412fa2d9f77a31b4789d (DEFAULT_TABLE table=test-table [id=fb020eb59e91443b8cdcee9614dbeece]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.276084 8128 tablet_service.cc:1467] Processing CreateTablet for tablet 29508b468330412fa2d9f77a31b4789d (DEFAULT_TABLE table=test-table [id=fb020eb59e91443b8cdcee9614dbeece]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.276505 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29508b468330412fa2d9f77a31b4789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.276809 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29508b468330412fa2d9f77a31b4789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.277050 8128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29508b468330412fa2d9f77a31b4789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.283926 8238 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Bootstrap starting.
I20260811 05:09:00.285017 8239 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Bootstrap starting.
I20260811 05:09:00.285929 8241 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Bootstrap starting.
I20260811 05:09:00.286190 8238 tablet_bootstrap.cc:654] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.287108 8238 log.cc:824] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:00.287153 8239 tablet_bootstrap.cc:654] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.287942 8239 log.cc:824] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:00.288297 8241 tablet_bootstrap.cc:654] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.289044 8238 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: No bootstrap required, opened a new log
I20260811 05:09:00.289043 8241 log.cc:824] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:00.289244 8238 ts_tablet_manager.cc:1397] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 05:09:00.289963 8239 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: No bootstrap required, opened a new log
I20260811 05:09:00.290151 8239 ts_tablet_manager.cc:1397] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:09:00.290944 8241 tablet_bootstrap.cc:492] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: No bootstrap required, opened a new log
I20260811 05:09:00.291276 8241 ts_tablet_manager.cc:1397] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:09:00.293059 8238 raft_consensus.cc:348] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.293277 8238 raft_consensus.cc:374] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.293327 8238 raft_consensus.cc:729] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd147a500d8419ca0232fd31536c088, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.293717 8238 consensus_queue.cc:260] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.293915 8239 raft_consensus.cc:348] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.294166 8239 raft_consensus.cc:374] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.294234 8239 raft_consensus.cc:729] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42577ab07f53476f94605c701146acbe, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.294518 8044 heartbeater.cc:503] Master 127.30.88.254:34311 was elected leader, sending a full tablet report...
I20260811 05:09:00.294631 8239 consensus_queue.cc:260] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.294719 8238 ts_tablet_manager.cc:1428] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260811 05:09:00.295297 8207 heartbeater.cc:503] Master 127.30.88.254:34311 was elected leader, sending a full tablet report...
I20260811 05:09:00.295591 8239 ts_tablet_manager.cc:1428] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:09:00.295696 8241 raft_consensus.cc:348] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.295933 8241 raft_consensus.cc:374] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.295982 8241 raft_consensus.cc:729] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.296406 8241 consensus_queue.cc:260] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.297012 7878 heartbeater.cc:503] Master 127.30.88.254:34311 was elected leader, sending a full tablet report...
I20260811 05:09:00.297260 8241 ts_tablet_manager.cc:1428] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:09:00.316495 8248 raft_consensus.cc:482] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:00.316661 8248 raft_consensus.cc:504] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.317524 8248 leader_election.cc:288] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42577ab07f53476f94605c701146acbe (127.30.88.195:34449), efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501)
I20260811 05:09:00.321169 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.317784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:54989 (local address 127.30.88.195:34449)
0811 05:09:00.317895 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.317912 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.318084 (+ 172us) server_negotiation.cc:407] Connection header received
0811 05:09:00.318223 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.318226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.318266 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.318331 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.318612 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.319138 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320248 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.320625 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320756 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.320806 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.320979 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.321044 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:09:00.321218 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.317692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:34449 (local address 127.30.88.193:54989)
0811 05:09:00.317987 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.318002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.318135 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.318350 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.318352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.318366 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.318494 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.318516 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.319241 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.319244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.320123 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.320133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.320756 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.320852 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.320878 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.320962 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.321029 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":5,"spinlock_wait_cycles":2816,"thread_start_us":65,"threads_started":1}
I20260811 05:09:00.321218 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.317848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:58883 (local address 127.30.88.194:34501)
0811 05:09:00.317987 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.317992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.318140 (+ 148us) server_negotiation.cc:407] Connection header received
0811 05:09:00.318261 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.318268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.318301 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.318364 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.318635 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.319132 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320281 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.320673 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320795 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.320848 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.321037 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.321100 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:09:00.321318 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.317733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34501 (local address 127.30.88.193:58883)
0811 05:09:00.318046 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.318056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.318174 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.318384 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.318387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.318411 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.318534 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.318539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.319236 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.319241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.320153 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.320164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.320791 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.320794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.320913 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.320931 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.321021 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.321089 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":108,"thread_start_us":87,"threads_started":1}
I20260811 05:09:00.321775 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29508b468330412fa2d9f77a31b4789d" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe" is_pre_election: true
I20260811 05:09:00.321807 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29508b468330412fa2d9f77a31b4789d" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088" is_pre_election: true
I20260811 05:09:00.321978 7991 raft_consensus.cc:2454] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.322016 8150 raft_consensus.cc:2454] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.322414 7753 leader_election.cc:302] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.322815 8248 raft_consensus.cc:2781] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:00.322913 8248 raft_consensus.cc:482] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:00.322981 8248 raft_consensus.cc:3037] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.323881 8248 raft_consensus.cc:504] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.324136 8248 leader_election.cc:288] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Requested vote from peers 42577ab07f53476f94605c701146acbe (127.30.88.195:34449), efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501)
I20260811 05:09:00.324481 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29508b468330412fa2d9f77a31b4789d" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe"
I20260811 05:09:00.324465 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29508b468330412fa2d9f77a31b4789d" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088"
I20260811 05:09:00.324575 7991 raft_consensus.cc:3037] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.324565 8150 raft_consensus.cc:3037] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.325484 7991 raft_consensus.cc:2454] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.325556 8150 raft_consensus.cc:2454] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.325806 7753 leader_election.cc:302] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.325984 8248 raft_consensus.cc:2781] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:00.326164 8248 raft_consensus.cc:686] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Becoming Leader. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Running, Role: LEADER
I20260811 05:09:00.326473 8248 consensus_queue.cc:237] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } }
I20260811 05:09:00.329268 7645 catalog_manager.cc:5495] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7305752aebfa457ea6029a0e18656837 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "7305752aebfa457ea6029a0e18656837" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } health_report { overall_health: UNKNOWN } } }
I20260811 05:09:00.347730 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.340579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.30.88.193:34953)
0811 05:09:00.343372 (+ 2793us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.343378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.343394 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:09:00.343495 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.343500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.343588 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.343711 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:00.343999 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.344641 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.345836 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.346166 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.346296 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.346608 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.347124 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.347226 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.347595 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.347639 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2691}
I20260811 05:09:00.354265 7991 raft_consensus.cc:1261] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.354624 8150 raft_consensus.cc:1261] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.354929 8248 consensus_queue.cc:1035] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.355247 8251 consensus_queue.cc:1035] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.363361 8254 mvcc.cc:205] Tried to move back new op lower bound from 7317196555684331520 to 7317196555579854848. Current Snapshot: MvccSnapshot[applied={T|T < 7317196555684331520}]
I20260811 05:09:00.364886 8258 mvcc.cc:205] Tried to move back new op lower bound from 7317196555684331520 to 7317196555579854848. Current Snapshot: MvccSnapshot[applied={T|T < 7317196555684331520}]
I20260811 05:09:00.371520 7644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.371706 7644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.375051 7644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Sending DeleteTablet for 3 replicas of tablet 29508b468330412fa2d9f77a31b4789d
I20260811 05:09:00.375804 8128 tablet_service.cc:1514] Processing DeleteTablet for tablet 29508b468330412fa2d9f77a31b4789d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:35670
I20260811 05:09:00.375799 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet 29508b468330412fa2d9f77a31b4789d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260811 05:09:00.375988 7967 tablet_service.cc:1514] Processing DeleteTablet for tablet 29508b468330412fa2d9f77a31b4789d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44714
I20260811 05:09:00.376475 8267 tablet_replica.cc:330] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: stopping tablet replica
I20260811 05:09:00.376822 8267 raft_consensus.cc:2229] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.377161 8267 raft_consensus.cc:2258] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.377112 7644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:09:00.377521 7644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:00.378738 8267 ts_tablet_manager.cc:1905] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.381616 8267 ts_tablet_manager.cc:1918] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.381780 8267 log.cc:1196] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/29508b468330412fa2d9f77a31b4789d
I20260811 05:09:00.382184 8267 ts_tablet_manager.cc:1939] T 29508b468330412fa2d9f77a31b4789d P efd147a500d8419ca0232fd31536c088: Deleting consensus metadata
I20260811 05:09:00.382519 7967 tablet_service.cc:1467] Processing CreateTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 (DEFAULT_TABLE table=test-table [id=2b34d66c6e5742ccb80320b9bc059030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.382582 8128 tablet_service.cc:1467] Processing CreateTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 (DEFAULT_TABLE table=test-table [id=2b34d66c6e5742ccb80320b9bc059030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.382791 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aca2ab8602475fbd5b09ad542ab3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.383081 8128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aca2ab8602475fbd5b09ad542ab3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.383358 7804 tablet_service.cc:1467] Processing CreateTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 (DEFAULT_TABLE table=test-table [id=2b34d66c6e5742ccb80320b9bc059030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.383620 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8aca2ab8602475fbd5b09ad542ab3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.384819 8238 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Bootstrap starting.
I20260811 05:09:00.385027 7629 catalog_manager.cc:4841] TS efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501): tablet 29508b468330412fa2d9f77a31b4789d (table test-table [id=fb020eb59e91443b8cdcee9614dbeece]) successfully deleted
I20260811 05:09:00.385623 8239 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Bootstrap starting.
I20260811 05:09:00.385780 8238 tablet_bootstrap.cc:654] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.386173 8241 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Bootstrap starting.
I20260811 05:09:00.386572 8239 tablet_bootstrap.cc:654] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.387187 8241 tablet_bootstrap.cc:654] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.376251 8265 tablet_replica.cc:330] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: stopping tablet replica
I20260811 05:09:00.388435 8239 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: No bootstrap required, opened a new log
I20260811 05:09:00.388510 8239 ts_tablet_manager.cc:1397] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:09:00.388500 8265 raft_consensus.cc:2229] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:09:00.388850 8265 raft_consensus.cc:2258] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.388975 8239 raft_consensus.cc:348] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.389083 8239 raft_consensus.cc:374] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.389128 8239 raft_consensus.cc:729] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42577ab07f53476f94605c701146acbe, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.389184 8266 tablet_replica.cc:330] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: stopping tablet replica
I20260811 05:09:00.389246 8239 consensus_queue.cc:260] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.389487 8239 ts_tablet_manager.cc:1428] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.389513 8266 raft_consensus.cc:2229] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.390005 8266 raft_consensus.cc:2258] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.390177 8265 ts_tablet_manager.cc:1905] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.391153 8241 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: No bootstrap required, opened a new log
I20260811 05:09:00.391264 8241 ts_tablet_manager.cc:1397] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 05:09:00.391673 8241 raft_consensus.cc:348] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.391772 8241 raft_consensus.cc:374] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.391800 8241 raft_consensus.cc:729] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.391909 8241 consensus_queue.cc:260] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.392004 8238 tablet_bootstrap.cc:492] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: No bootstrap required, opened a new log
I20260811 05:09:00.392081 8238 ts_tablet_manager.cc:1397] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260811 05:09:00.392115 8241 ts_tablet_manager.cc:1428] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.392553 8238 raft_consensus.cc:348] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.392652 8238 raft_consensus.cc:374] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.392680 8238 raft_consensus.cc:729] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd147a500d8419ca0232fd31536c088, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.392784 8238 consensus_queue.cc:260] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.393038 8238 ts_tablet_manager.cc:1428] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:09:00.393240 8265 ts_tablet_manager.cc:1918] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.393342 8265 log.cc:1196] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/29508b468330412fa2d9f77a31b4789d
I20260811 05:09:00.393672 8265 ts_tablet_manager.cc:1939] T 29508b468330412fa2d9f77a31b4789d P 7305752aebfa457ea6029a0e18656837: Deleting consensus metadata
I20260811 05:09:00.395149 7627 catalog_manager.cc:4841] TS 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953): tablet 29508b468330412fa2d9f77a31b4789d (table test-table [id=fb020eb59e91443b8cdcee9614dbeece]) successfully deleted
I20260811 05:09:00.396412 8266 ts_tablet_manager.cc:1905] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.399075 8266 ts_tablet_manager.cc:1918] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.399223 8266 log.cc:1196] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/29508b468330412fa2d9f77a31b4789d
I20260811 05:09:00.399607 8266 ts_tablet_manager.cc:1939] T 29508b468330412fa2d9f77a31b4789d P 42577ab07f53476f94605c701146acbe: Deleting consensus metadata
I20260811 05:09:00.400780 7628 catalog_manager.cc:4841] TS 42577ab07f53476f94605c701146acbe (127.30.88.195:34449): tablet 29508b468330412fa2d9f77a31b4789d (table test-table [id=fb020eb59e91443b8cdcee9614dbeece]) successfully deleted
W20260811 05:09:00.458709 8208 tablet.cc:2369] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:09:00.458781 8248 raft_consensus.cc:482] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:00.458948 8248 raft_consensus.cc:504] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.459471 8248 leader_election.cc:288] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.459852 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aca2ab8602475fbd5b09ad542ab3e6" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088" is_pre_election: true
I20260811 05:09:00.460003 7991 raft_consensus.cc:2454] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.459982 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aca2ab8602475fbd5b09ad542ab3e6" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe" is_pre_election: true
I20260811 05:09:00.460136 8150 raft_consensus.cc:2454] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.460316 7753 leader_election.cc:302] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.460526 8248 raft_consensus.cc:2781] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:00.460598 8248 raft_consensus.cc:482] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:00.460645 8248 raft_consensus.cc:3037] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.461841 8248 raft_consensus.cc:504] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.462159 8248 leader_election.cc:288] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Requested vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.462461 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aca2ab8602475fbd5b09ad542ab3e6" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe"
I20260811 05:09:00.462457 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8aca2ab8602475fbd5b09ad542ab3e6" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088"
I20260811 05:09:00.462580 7991 raft_consensus.cc:3037] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.462580 8150 raft_consensus.cc:3037] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.463766 8150 raft_consensus.cc:2454] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.463768 7991 raft_consensus.cc:2454] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.464046 7753 leader_election.cc:302] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.464293 8248 raft_consensus.cc:2781] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:00.464406 8248 raft_consensus.cc:686] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Becoming Leader. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Running, Role: LEADER
I20260811 05:09:00.464555 8248 consensus_queue.cc:237] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.466373 7645 catalog_manager.cc:5495] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7305752aebfa457ea6029a0e18656837 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "7305752aebfa457ea6029a0e18656837" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } health_report { overall_health: UNKNOWN } } }
W20260811 05:09:00.490276 8045 tablet.cc:2369] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424939, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.254@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:09:00.501295 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.492543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34311 (local address 127.0.0.1:42744)
0811 05:09:00.492830 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.492847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.492948 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.493210 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.493212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.493559 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:00.493650 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.493655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.494586 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.494588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.495165 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.495173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.495734 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.495737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.495808 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.496405 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:00.496416 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:00.497711 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:00.499769 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.499778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.499782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.500010 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.500322 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:00.500324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:00.500326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:00.500421 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:00.500859 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:00.500866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:00.500964 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.501057 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.501164 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":135,"threads_started":1}
I20260811 05:09:00.501457 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.492589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.30.88.254:34311)
0811 05:09:00.492726 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.492732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.492921 (+ 189us) server_negotiation.cc:407] Connection header received
0811 05:09:00.493035 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.493038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.493099 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.493193 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.493767 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.494487 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.495279 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.495622 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.495739 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.497851 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:00.497855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:00.497858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:00.497882 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:00.499645 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.500115 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.500119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.500121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.500218 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.500513 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.500517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.500519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.500750 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.500862 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.501148 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.501275 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:09:00.505369 7645 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:42744:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:09:00.506058 7645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:00.511400 7804 tablet_service.cc:1467] Processing CreateTablet for tablet f1a4cb73f21f42109670df5e6c7b16c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a41975d8cdb4abe97d31c43e309219f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:09:00.511842 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a4cb73f21f42109670df5e6c7b16c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.511662 8128 tablet_service.cc:1467] Processing CreateTablet for tablet f1a4cb73f21f42109670df5e6c7b16c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a41975d8cdb4abe97d31c43e309219f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:09:00.511708 7967 tablet_service.cc:1467] Processing CreateTablet for tablet f1a4cb73f21f42109670df5e6c7b16c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a41975d8cdb4abe97d31c43e309219f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:09:00.512076 8128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a4cb73f21f42109670df5e6c7b16c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.512120 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a4cb73f21f42109670df5e6c7b16c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.514819 8241 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: Bootstrap starting.
I20260811 05:09:00.514992 8239 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: Bootstrap starting.
I20260811 05:09:00.515368 8238 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: Bootstrap starting.
I20260811 05:09:00.515988 8239 tablet_bootstrap.cc:654] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.515988 8241 tablet_bootstrap.cc:654] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.516317 8238 tablet_bootstrap.cc:654] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.517588 8238 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: No bootstrap required, opened a new log
I20260811 05:09:00.517676 8238 ts_tablet_manager.cc:1397] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:09:00.518085 8238 raft_consensus.cc:348] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.518198 8238 raft_consensus.cc:374] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.518237 8238 raft_consensus.cc:729] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd147a500d8419ca0232fd31536c088, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.518348 8238 consensus_queue.cc:260] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.518635 8238 ts_tablet_manager.cc:1428] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:09:00.518823 8246 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.519515 8246 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.520581 8239 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: No bootstrap required, opened a new log
I20260811 05:09:00.520669 8241 tablet_bootstrap.cc:492] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: No bootstrap required, opened a new log
I20260811 05:09:00.520750 8241 ts_tablet_manager.cc:1397] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260811 05:09:00.520805 8239 ts_tablet_manager.cc:1397] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 05:09:00.521222 8241 raft_consensus.cc:348] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.521361 8241 raft_consensus.cc:374] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.521390 8241 raft_consensus.cc:729] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.521503 8241 consensus_queue.cc:260] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.521639 8239 raft_consensus.cc:348] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.521751 8239 raft_consensus.cc:374] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.521785 8239 raft_consensus.cc:729] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42577ab07f53476f94605c701146acbe, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.521734 8248 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.521847 8248 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.521889 8239 consensus_queue.cc:260] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.522148 8239 ts_tablet_manager.cc:1428] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.522550 8247 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.522656 8241 ts_tablet_manager.cc:1428] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:09:00.522675 8247 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 05:09:00.532375 7879 tablet.cc:2369] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:09:00.541771 8246 raft_consensus.cc:482] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:00.541915 8246 raft_consensus.cc:504] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.543092 8246 leader_election.cc:288] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.547902 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.543600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:56977 (local address 127.30.88.193:34953)
0811 05:09:00.543713 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.543718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.543821 (+ 103us) server_negotiation.cc:407] Connection header received
0811 05:09:00.544002 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.544005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.544046 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.544108 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.544684 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.545425 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.546812 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.547258 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.547386 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.547440 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.547673 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.547749 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:09:00.548141 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.543433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:55941 (local address 127.30.88.195:34449)
0811 05:09:00.543566 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.543571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.543695 (+ 124us) server_negotiation.cc:407] Connection header received
0811 05:09:00.543973 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.543976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.544030 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.544083 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:00.544665 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.545163 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.546735 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.547391 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.547503 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.547549 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.547867 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.547989 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:09:00.548332 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.543282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:34449 (local address 127.30.88.194:55941)
0811 05:09:00.543611 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.543625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.543774 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.544248 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.544251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.544267 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.544527 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.544537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.545263 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.545268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.546607 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.546618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.547607 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.547610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.547713 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.547735 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.547846 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.547913 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":41,"spinlock_wait_cycles":2176,"thread_start_us":97,"threads_started":1}
I20260811 05:09:00.548352 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.543282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34953 (local address 127.30.88.194:56977)
0811 05:09:00.543727 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.543739 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.543871 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.544270 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.544272 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.544305 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:00.544537 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.544579 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.545531 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.545534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.546656 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.546665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.547379 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.547383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.547502 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.547522 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.548118 (+ 596us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.548204 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":150,"spinlock_wait_cycles":22528,"thread_start_us":60,"threads_started":1}
I20260811 05:09:00.548954 7825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a4cb73f21f42109670df5e6c7b16c1" candidate_uuid: "efd147a500d8419ca0232fd31536c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7305752aebfa457ea6029a0e18656837" is_pre_election: true
I20260811 05:09:00.549144 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a4cb73f21f42109670df5e6c7b16c1" candidate_uuid: "efd147a500d8419ca0232fd31536c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe" is_pre_election: true
I20260811 05:09:00.549171 7825 raft_consensus.cc:2454] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd147a500d8419ca0232fd31536c088 in term 0.
I20260811 05:09:00.549279 8150 raft_consensus.cc:2454] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd147a500d8419ca0232fd31536c088 in term 0.
I20260811 05:09:00.549654 7910 leader_election.cc:302] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42577ab07f53476f94605c701146acbe, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.549901 8246 raft_consensus.cc:2781] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:00.549979 8246 raft_consensus.cc:482] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:00.550026 8246 raft_consensus.cc:3037] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.551337 8246 raft_consensus.cc:504] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.551632 8246 leader_election.cc:288] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [CANDIDATE]: Term 1 election: Requested vote from peers 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.551954 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a4cb73f21f42109670df5e6c7b16c1" candidate_uuid: "efd147a500d8419ca0232fd31536c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe"
I20260811 05:09:00.552055 8150 raft_consensus.cc:3037] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.553638 8150 raft_consensus.cc:2454] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd147a500d8419ca0232fd31536c088 in term 1.
I20260811 05:09:00.553855 7825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a4cb73f21f42109670df5e6c7b16c1" candidate_uuid: "efd147a500d8419ca0232fd31536c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7305752aebfa457ea6029a0e18656837"
I20260811 05:09:00.553880 7910 leader_election.cc:302] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42577ab07f53476f94605c701146acbe, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.553967 7825 raft_consensus.cc:3037] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.554066 8246 raft_consensus.cc:2781] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:00.554248 8246 raft_consensus.cc:686] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [term 1 LEADER]: Becoming Leader. State: Replica: efd147a500d8419ca0232fd31536c088, State: Running, Role: LEADER
I20260811 05:09:00.554469 8246 consensus_queue.cc:237] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.554973 7825 raft_consensus.cc:2454] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd147a500d8419ca0232fd31536c088 in term 1.
I20260811 05:09:00.555603 8286 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: TxnStatusTablet state changed. Reason: New leader efd147a500d8419ca0232fd31536c088. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.555683 8286 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:09:00.555920 8288 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:09:00.556022 8288 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:09:00.556777 7645 catalog_manager.cc:5495] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd147a500d8419ca0232fd31536c088 (127.30.88.194). New cstate: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } health_report { overall_health: UNKNOWN } } }
I20260811 05:09:00.567981 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.564191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.30.88.194:34501)
0811 05:09:00.564295 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.564299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.564408 (+ 109us) server_negotiation.cc:407] Connection header received
0811 05:09:00.564562 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.564563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.564607 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.564678 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:00.564932 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.565491 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.566722 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.566954 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.567023 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.567320 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.567646 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.567722 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.567837 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.567896 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:09:00.568056 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.564164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34501 (local address 127.0.0.1:44726)
0811 05:09:00.564320 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.564335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.564445 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.564715 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.564718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.564726 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:09:00.564828 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.564834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.565590 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.565593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.566599 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.566610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.567051 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.567055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.567142 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.567194 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.567734 (+ 540us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.567748 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.567820 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.567918 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 05:09:00.570722 8150 raft_consensus.cc:1261] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Refusing update from remote peer efd147a500d8419ca0232fd31536c088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.570772 7825 raft_consensus.cc:1261] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Refusing update from remote peer efd147a500d8419ca0232fd31536c088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.571245 8286 consensus_queue.cc:1035] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.571435 8246 consensus_queue.cc:1035] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.572716 8247 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: TxnStatusTablet state changed. Reason: New leader efd147a500d8419ca0232fd31536c088. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.572813 8247 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.573665 8248 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: TxnStatusTablet state changed. Reason: New leader efd147a500d8419ca0232fd31536c088. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.573760 8248 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.575353 8286 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.575471 8286 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:09:00.575594 8286 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.575671 8286 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P efd147a500d8419ca0232fd31536c088: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:09:00.576295 8255 mvcc.cc:205] Tried to move back new op lower bound from 7317196556573032448 to 7317196556513427456. Current Snapshot: MvccSnapshot[applied={T|T < 7317196556573032448 or (T in {7317196556573032448})}]
I20260811 05:09:00.576421 8247 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.576534 8247 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 42577ab07f53476f94605c701146acbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.577459 8248 tablet_replica.cc:439] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efd147a500d8419ca0232fd31536c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } } }
I20260811 05:09:00.577564 8248 tablet_replica.cc:442] T f1a4cb73f21f42109670df5e6c7b16c1 P 7305752aebfa457ea6029a0e18656837: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:09:00.590534 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.587186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:44445 (local address 127.30.88.194:34501)
0811 05:09:00.587298 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.587302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.587538 (+ 236us) server_negotiation.cc:407] Connection header received
0811 05:09:00.587701 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.587704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.587753 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.587826 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:00.588072 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.588562 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.589455 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.589684 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.589785 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.590064 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.590142 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.590218 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.590333 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.590406 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:09:00.590539 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.587099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34501 (local address 127.30.88.193:44445)
0811 05:09:00.587450 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.587463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.587593 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.587848 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.587851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.587860 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:09:00.587969 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.587974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.588672 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.588676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.589315 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.589324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.589797 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.589800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.589885 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.589934 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.590235 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.590242 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.590320 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.590412 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260811 05:09:00.597450 7991 raft_consensus.cc:1261] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.597534 8150 raft_consensus.cc:1261] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.597878 8251 consensus_queue.cc:1035] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.598270 8251 consensus_queue.cc:1035] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.620187 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.616397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34953 (local address 127.30.88.194:54493)
0811 05:09:00.616736 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:00.616752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:00.616869 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:00.617082 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:00.617085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:00.617092 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:09:00.617223 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.617229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.617952 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.617956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.618741 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:00.618751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.619437 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.619441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:00.619518 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.619562 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.619902 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.619909 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:09:00.619981 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:09:00.620049 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":136,"threads_started":1}
I20260811 05:09:00.620193 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.616440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:54493 (local address 127.30.88.193:34953)
0811 05:09:00.616586 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.616591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.616834 (+ 243us) server_negotiation.cc:407] Connection header received
0811 05:09:00.616949 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.616951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.616993 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.617058 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:00.617313 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.617840 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.618890 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.619228 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.619333 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.619699 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.619785 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.619871 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.620012 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.620071 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:09:00.643935 7645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.644102 7645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.646136 7645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Sending DeleteTablet for 3 replicas of tablet c8aca2ab8602475fbd5b09ad542ab3e6
I20260811 05:09:00.646649 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260811 05:09:00.646832 8265 tablet_replica.cc:330] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: stopping tablet replica
I20260811 05:09:00.646895 8128 tablet_service.cc:1514] Processing DeleteTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:35670
I20260811 05:09:00.646977 8265 raft_consensus.cc:2229] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:09:00.646957 7967 tablet_service.cc:1514] Processing DeleteTablet for tablet c8aca2ab8602475fbd5b09ad542ab3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44714
I20260811 05:09:00.647138 8266 tablet_replica.cc:330] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: stopping tablet replica
I20260811 05:09:00.647153 8265 raft_consensus.cc:2258] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.647230 8267 tablet_replica.cc:330] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: stopping tablet replica
I20260811 05:09:00.647275 8266 raft_consensus.cc:2229] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.647329 8267 raft_consensus.cc:2229] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.647385 8266 raft_consensus.cc:2258] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.647423 8267 raft_consensus.cc:2258] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.648270 8266 ts_tablet_manager.cc:1905] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.650584 8267 ts_tablet_manager.cc:1905] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.651515 8265 ts_tablet_manager.cc:1905] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.653290 8267 ts_tablet_manager.cc:1918] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:00.653367 8267 log.cc:1196] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/c8aca2ab8602475fbd5b09ad542ab3e6
I20260811 05:09:00.653602 8267 ts_tablet_manager.cc:1939] T c8aca2ab8602475fbd5b09ad542ab3e6 P efd147a500d8419ca0232fd31536c088: Deleting consensus metadata
I20260811 05:09:00.654307 8265 ts_tablet_manager.cc:1918] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:00.654403 8265 log.cc:1196] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/c8aca2ab8602475fbd5b09ad542ab3e6
I20260811 05:09:00.654629 8265 ts_tablet_manager.cc:1939] T c8aca2ab8602475fbd5b09ad542ab3e6 P 7305752aebfa457ea6029a0e18656837: Deleting consensus metadata
I20260811 05:09:00.654927 7629 catalog_manager.cc:4841] TS efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501): tablet c8aca2ab8602475fbd5b09ad542ab3e6 (table test-table [id=2b34d66c6e5742ccb80320b9bc059030]) successfully deleted
I20260811 05:09:00.655668 8266 ts_tablet_manager.cc:1918] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:00.655761 8266 log.cc:1196] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/c8aca2ab8602475fbd5b09ad542ab3e6
I20260811 05:09:00.656008 8266 ts_tablet_manager.cc:1939] T c8aca2ab8602475fbd5b09ad542ab3e6 P 42577ab07f53476f94605c701146acbe: Deleting consensus metadata
I20260811 05:09:00.656661 7627 catalog_manager.cc:4841] TS 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953): tablet c8aca2ab8602475fbd5b09ad542ab3e6 (table test-table [id=2b34d66c6e5742ccb80320b9bc059030]) successfully deleted
I20260811 05:09:00.656731 7628 catalog_manager.cc:4841] TS 42577ab07f53476f94605c701146acbe (127.30.88.195:34449): tablet c8aca2ab8602475fbd5b09ad542ab3e6 (table test-table [id=2b34d66c6e5742ccb80320b9bc059030]) successfully deleted
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:09:00.681078 7700 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:00.674129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.30.88.254:34311)
0811 05:09:00.674270 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.674276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.674418 (+ 142us) server_negotiation.cc:407] Connection header received
0811 05:09:00.674568 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.674574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.674634 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.674720 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.675539 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.676394 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.677325 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.677693 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.677810 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.680945 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42750: BlockingRecv error: recv got EOF from 127.0.0.1:42750 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 05:09:00 dist-test-slave-7c35 krb5kdc[7595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM
I20260811 05:09:00.691105 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.682407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.30.88.254:34311)
0811 05:09:00.682533 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.682536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.682635 (+ 99us) server_negotiation.cc:407] Connection header received
0811 05:09:00.682767 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.682771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.682817 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.683015 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:00.683568 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.684098 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.684977 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.685220 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.685343 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.687780 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:00.687783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:00.687785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:00.687807 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:00.689273 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.689762 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.689764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.689765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.689837 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:00.690117 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:00.690119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:00.690120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:00.690284 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:00.690381 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.690819 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.690967 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:09:00.693604 7645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:09:00.694054 7645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:00.698850 7804 tablet_service.cc:1467] Processing CreateTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 (DEFAULT_TABLE table=test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.699255 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3dc70428510406ba20eaa5111c1c3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.699218 8128 tablet_service.cc:1467] Processing CreateTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 (DEFAULT_TABLE table=test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.699486 8128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3dc70428510406ba20eaa5111c1c3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.701768 8241 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Bootstrap starting.
I20260811 05:09:00.701967 7967 tablet_service.cc:1467] Processing CreateTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 (DEFAULT_TABLE table=test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.702241 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3dc70428510406ba20eaa5111c1c3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.702713 8241 tablet_bootstrap.cc:654] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.703028 8239 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Bootstrap starting.
I20260811 05:09:00.703858 8241 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: No bootstrap required, opened a new log
I20260811 05:09:00.703939 8241 ts_tablet_manager.cc:1397] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:09:00.703955 8239 tablet_bootstrap.cc:654] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.704345 8241 raft_consensus.cc:348] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.704459 8241 raft_consensus.cc:374] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.704492 8241 raft_consensus.cc:729] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.704627 8241 consensus_queue.cc:260] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.704933 8241 ts_tablet_manager.cc:1428] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.705092 8238 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Bootstrap starting.
I20260811 05:09:00.706101 8238 tablet_bootstrap.cc:654] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.706348 8239 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: No bootstrap required, opened a new log
I20260811 05:09:00.706419 8239 ts_tablet_manager.cc:1397] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:09:00.706925 8239 raft_consensus.cc:348] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.707069 8239 raft_consensus.cc:374] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.707109 8239 raft_consensus.cc:729] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42577ab07f53476f94605c701146acbe, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.707227 8239 consensus_queue.cc:260] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.707502 8239 ts_tablet_manager.cc:1428] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.707508 8238 tablet_bootstrap.cc:492] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: No bootstrap required, opened a new log
I20260811 05:09:00.707589 8238 ts_tablet_manager.cc:1397] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 05:09:00.708032 8238 raft_consensus.cc:348] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.708137 8238 raft_consensus.cc:374] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.708171 8238 raft_consensus.cc:729] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd147a500d8419ca0232fd31536c088, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.708288 8238 consensus_queue.cc:260] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.708549 8238 ts_tablet_manager.cc:1428] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:09:00.886731 8300 raft_consensus.cc:482] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:00.886917 8300 raft_consensus.cc:504] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.887389 8300 leader_election.cc:288] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.887780 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3dc70428510406ba20eaa5111c1c3e5" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe" is_pre_election: true
I20260811 05:09:00.887780 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3dc70428510406ba20eaa5111c1c3e5" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088" is_pre_election: true
I20260811 05:09:00.887923 8150 raft_consensus.cc:2454] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.887923 7991 raft_consensus.cc:2454] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:00.888206 7752 leader_election.cc:302] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42577ab07f53476f94605c701146acbe, 7305752aebfa457ea6029a0e18656837; no voters:
I20260811 05:09:00.888397 8300 raft_consensus.cc:2781] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:00.888464 8300 raft_consensus.cc:482] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:00.888509 8300 raft_consensus.cc:3037] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.889485 8300 raft_consensus.cc:504] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.889777 8300 leader_election.cc:288] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Requested vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:00.890053 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3dc70428510406ba20eaa5111c1c3e5" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088"
I20260811 05:09:00.890105 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3dc70428510406ba20eaa5111c1c3e5" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe"
I20260811 05:09:00.890215 7991 raft_consensus.cc:3037] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.890288 8150 raft_consensus.cc:3037] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:00.891515 7991 raft_consensus.cc:2454] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.891515 8150 raft_consensus.cc:2454] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:00.891789 7753 leader_election.cc:302] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:00.891986 8300 raft_consensus.cc:2781] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:00.892097 8300 raft_consensus.cc:686] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Becoming Leader. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Running, Role: LEADER
I20260811 05:09:00.892266 8300 consensus_queue.cc:237] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.893780 7644 catalog_manager.cc:5495] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7305752aebfa457ea6029a0e18656837 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "7305752aebfa457ea6029a0e18656837" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } health_report { overall_health: UNKNOWN } } }
I20260811 05:09:00.939980 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:00.937003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57800 (local address 127.30.88.193:34953)
0811 05:09:00.937139 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:00.937143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:00.937157 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:00.937240 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:00.937242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:00.937289 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:00.937352 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:00.937623 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.938202 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.939076 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:00.939271 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:00.939345 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:00.939566 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:00.939629 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:00.939680 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:00.939755 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 05:09:00.939801 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:09:00.941980 8150 raft_consensus.cc:1261] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.941979 7991 raft_consensus.cc:1261] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:00.942389 8300 consensus_queue.cc:1035] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.942523 8251 consensus_queue.cc:1035] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:00.949693 7643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.949803 7643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:00.951671 7643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Sending DeleteTablet for 3 replicas of tablet f3dc70428510406ba20eaa5111c1c3e5
I20260811 05:09:00.952199 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260811 05:09:00.952248 8128 tablet_service.cc:1514] Processing DeleteTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:35670
I20260811 05:09:00.952373 8265 tablet_replica.cc:330] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: stopping tablet replica
I20260811 05:09:00.952381 7967 tablet_service.cc:1514] Processing DeleteTablet for tablet f3dc70428510406ba20eaa5111c1c3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:00 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44714
I20260811 05:09:00.952410 8266 tablet_replica.cc:330] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: stopping tablet replica
I20260811 05:09:00.952509 8265 raft_consensus.cc:2229] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:09:00.952534 8267 tablet_replica.cc:330] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: stopping tablet replica
I20260811 05:09:00.952527 8266 raft_consensus.cc:2229] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.952628 8267 raft_consensus.cc:2229] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:00.952643 8266 raft_consensus.cc:2258] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.952675 8265 raft_consensus.cc:2258] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.952728 8267 raft_consensus.cc:2258] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:00.953377 8266 ts_tablet_manager.cc:1905] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.953382 8265 ts_tablet_manager.cc:1905] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.953586 8267 ts_tablet_manager.cc:1905] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:00.955416 8265 ts_tablet_manager.cc:1918] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.955508 8265 log.cc:1196] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/f3dc70428510406ba20eaa5111c1c3e5
I20260811 05:09:00.955772 8265 ts_tablet_manager.cc:1939] T f3dc70428510406ba20eaa5111c1c3e5 P 7305752aebfa457ea6029a0e18656837: Deleting consensus metadata
I20260811 05:09:00.955865 8266 ts_tablet_manager.cc:1918] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.955859 8267 ts_tablet_manager.cc:1918] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:00.955946 8266 log.cc:1196] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/f3dc70428510406ba20eaa5111c1c3e5
I20260811 05:09:00.955955 8267 log.cc:1196] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/f3dc70428510406ba20eaa5111c1c3e5
I20260811 05:09:00.956216 8266 ts_tablet_manager.cc:1939] T f3dc70428510406ba20eaa5111c1c3e5 P 42577ab07f53476f94605c701146acbe: Deleting consensus metadata
I20260811 05:09:00.956216 8267 ts_tablet_manager.cc:1939] T f3dc70428510406ba20eaa5111c1c3e5 P efd147a500d8419ca0232fd31536c088: Deleting consensus metadata
I20260811 05:09:00.956405 7627 catalog_manager.cc:4841] TS 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953): tablet f3dc70428510406ba20eaa5111c1c3e5 (table test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]) successfully deleted
I20260811 05:09:00.956509 7643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:09:00.956832 7629 catalog_manager.cc:4841] TS efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501): tablet f3dc70428510406ba20eaa5111c1c3e5 (table test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]) successfully deleted
W20260811 05:09:00.956954 7643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:00.957669 7628 catalog_manager.cc:4841] TS 42577ab07f53476f94605c701146acbe (127.30.88.195:34449): tablet f3dc70428510406ba20eaa5111c1c3e5 (table test-table [id=93fbe9219f0c4dc18023ffcac3cbc491]) successfully deleted
I20260811 05:09:00.960875 7804 tablet_service.cc:1467] Processing CreateTablet for tablet d73fe7152edd46148420620b5055b85e (DEFAULT_TABLE table=test-table [id=5a1f307903764644866b838fb74ac441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.960925 7967 tablet_service.cc:1467] Processing CreateTablet for tablet d73fe7152edd46148420620b5055b85e (DEFAULT_TABLE table=test-table [id=5a1f307903764644866b838fb74ac441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.961017 8128 tablet_service.cc:1467] Processing CreateTablet for tablet d73fe7152edd46148420620b5055b85e (DEFAULT_TABLE table=test-table [id=5a1f307903764644866b838fb74ac441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:00.961112 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73fe7152edd46148420620b5055b85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.961184 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73fe7152edd46148420620b5055b85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.961230 8128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73fe7152edd46148420620b5055b85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:00.962685 8238 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Bootstrap starting.
I20260811 05:09:00.963176 8241 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Bootstrap starting.
I20260811 05:09:00.963514 8238 tablet_bootstrap.cc:654] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.963730 8241 tablet_bootstrap.cc:654] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.963995 8239 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Bootstrap starting.
I20260811 05:09:00.964692 8241 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: No bootstrap required, opened a new log
I20260811 05:09:00.964746 8241 ts_tablet_manager.cc:1397] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:09:00.964756 8239 tablet_bootstrap.cc:654] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:00.964998 8241 raft_consensus.cc:348] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.965068 8241 raft_consensus.cc:374] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.965087 8241 raft_consensus.cc:729] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.965220 8241 consensus_queue.cc:260] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.965561 8241 ts_tablet_manager.cc:1428] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:09:00.966378 8239 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: No bootstrap required, opened a new log
I20260811 05:09:00.966380 8238 tablet_bootstrap.cc:492] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: No bootstrap required, opened a new log
I20260811 05:09:00.966483 8239 ts_tablet_manager.cc:1397] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:09:00.966497 8238 ts_tablet_manager.cc:1397] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 05:09:00.966928 8239 raft_consensus.cc:348] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.967041 8239 raft_consensus.cc:374] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.966980 8238 raft_consensus.cc:348] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.967078 8239 raft_consensus.cc:729] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42577ab07f53476f94605c701146acbe, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.967087 8238 raft_consensus.cc:374] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:00.967118 8238 raft_consensus.cc:729] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd147a500d8419ca0232fd31536c088, State: Initialized, Role: FOLLOWER
I20260811 05:09:00.967196 8239 consensus_queue.cc:260] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.967247 8238 consensus_queue.cc:260] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:00.967515 8239 ts_tablet_manager.cc:1428] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:00.967545 8238 ts_tablet_manager.cc:1428] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:09:01.018383 8300 raft_consensus.cc:482] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:01.018517 8300 raft_consensus.cc:504] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:01.018976 8300 leader_election.cc:288] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:01.019266 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73fe7152edd46148420620b5055b85e" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088" is_pre_election: true
I20260811 05:09:01.019433 7991 raft_consensus.cc:2454] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:01.019410 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73fe7152edd46148420620b5055b85e" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe" is_pre_election: true
I20260811 05:09:01.019572 8150 raft_consensus.cc:2454] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 0.
I20260811 05:09:01.019757 7753 leader_election.cc:302] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:01.019950 8300 raft_consensus.cc:2781] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:01.020035 8300 raft_consensus.cc:482] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:01.020097 8300 raft_consensus.cc:3037] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:01.021031 8300 raft_consensus.cc:504] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:01.021319 8300 leader_election.cc:288] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Requested vote from peers efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501), 42577ab07f53476f94605c701146acbe (127.30.88.195:34449)
I20260811 05:09:01.021642 7991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73fe7152edd46148420620b5055b85e" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd147a500d8419ca0232fd31536c088"
I20260811 05:09:01.021677 8150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73fe7152edd46148420620b5055b85e" candidate_uuid: "7305752aebfa457ea6029a0e18656837" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42577ab07f53476f94605c701146acbe"
I20260811 05:09:01.021765 8150 raft_consensus.cc:3037] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:01.021765 7991 raft_consensus.cc:3037] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:01.022850 7991 raft_consensus.cc:2454] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:01.022886 8150 raft_consensus.cc:2454] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7305752aebfa457ea6029a0e18656837 in term 1.
I20260811 05:09:01.023152 7753 leader_election.cc:302] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7305752aebfa457ea6029a0e18656837, efd147a500d8419ca0232fd31536c088; no voters:
I20260811 05:09:01.023334 8300 raft_consensus.cc:2781] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:01.023460 8300 raft_consensus.cc:686] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Becoming Leader. State: Replica: 7305752aebfa457ea6029a0e18656837, State: Running, Role: LEADER
I20260811 05:09:01.023612 8300 consensus_queue.cc:237] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } }
I20260811 05:09:01.025198 7642 catalog_manager.cc:5495] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7305752aebfa457ea6029a0e18656837 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "7305752aebfa457ea6029a0e18656837" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7305752aebfa457ea6029a0e18656837" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 } health_report { overall_health: UNKNOWN } } }
I20260811 05:09:01.051818 8150 raft_consensus.cc:1261] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:01.052069 7991 raft_consensus.cc:1261] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Refusing update from remote peer 7305752aebfa457ea6029a0e18656837: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:01.052207 8327 consensus_queue.cc:1035] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42577ab07f53476f94605c701146acbe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:01.052410 8300 consensus_queue.cc:1035] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd147a500d8419ca0232fd31536c088" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:01.081275 7645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:01.081394 7645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:01.083179 7645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a54c94c2361847c88e8cbb49f5a0476b: Sending DeleteTablet for 3 replicas of tablet d73fe7152edd46148420620b5055b85e
I20260811 05:09:01.083703 7967 tablet_service.cc:1514] Processing DeleteTablet for tablet d73fe7152edd46148420620b5055b85e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:01 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44714
I20260811 05:09:01.083710 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet d73fe7152edd46148420620b5055b85e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:01 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260811 05:09:01.083730 8128 tablet_service.cc:1514] Processing DeleteTablet for tablet d73fe7152edd46148420620b5055b85e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:01 UTC) from {username='oryx', principal='oryx/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:35670
I20260811 05:09:01.083853 8267 tablet_replica.cc:330] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: stopping tablet replica
I20260811 05:09:01.083885 8266 tablet_replica.cc:330] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: stopping tablet replica
I20260811 05:09:01.083918 8265 tablet_replica.cc:330] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: stopping tablet replica
I20260811 05:09:01.083961 8267 raft_consensus.cc:2229] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:01.083976 8266 raft_consensus.cc:2229] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:01.084004 8265 raft_consensus.cc:2229] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:09:01.084054 8267 raft_consensus.cc:2258] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:01.084054 8266 raft_consensus.cc:2258] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:01.084120 8265 raft_consensus.cc:2258] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:01.084789 8265 ts_tablet_manager.cc:1905] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:01.084789 8267 ts_tablet_manager.cc:1905] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:01.084978 8266 ts_tablet_manager.cc:1905] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:01.086939 8265 ts_tablet_manager.cc:1918] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:01.087157 8265 log.cc:1196] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/d73fe7152edd46148420620b5055b85e
I20260811 05:09:01.087339 8266 ts_tablet_manager.cc:1918] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:01.087352 8267 ts_tablet_manager.cc:1918] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:09:01.087427 8267 log.cc:1196] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/d73fe7152edd46148420620b5055b85e
I20260811 05:09:01.087428 8265 ts_tablet_manager.cc:1939] T d73fe7152edd46148420620b5055b85e P 7305752aebfa457ea6029a0e18656837: Deleting consensus metadata
I20260811 05:09:01.087427 8266 log.cc:1196] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/d73fe7152edd46148420620b5055b85e
I20260811 05:09:01.087700 8267 ts_tablet_manager.cc:1939] T d73fe7152edd46148420620b5055b85e P efd147a500d8419ca0232fd31536c088: Deleting consensus metadata
I20260811 05:09:01.087700 8266 ts_tablet_manager.cc:1939] T d73fe7152edd46148420620b5055b85e P 42577ab07f53476f94605c701146acbe: Deleting consensus metadata
I20260811 05:09:01.088127 7627 catalog_manager.cc:4841] TS 7305752aebfa457ea6029a0e18656837 (127.30.88.193:34953): tablet d73fe7152edd46148420620b5055b85e (table test-table [id=5a1f307903764644866b838fb74ac441]) successfully deleted
I20260811 05:09:01.088339 7628 catalog_manager.cc:4841] TS 42577ab07f53476f94605c701146acbe (127.30.88.195:34449): tablet d73fe7152edd46148420620b5055b85e (table test-table [id=5a1f307903764644866b838fb74ac441]) successfully deleted
I20260811 05:09:01.088367 7629 catalog_manager.cc:4841] TS efd147a500d8419ca0232fd31536c088 (127.30.88.194:34501): tablet d73fe7152edd46148420620b5055b85e (table test-table [id=5a1f307903764644866b838fb74ac441]) successfully deleted
I20260811 05:09:01.088691 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 7729
I20260811 05:09:01.100173 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 7887
I20260811 05:09:01.108592 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8053
I20260811 05:09:01.115589 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 7610
2026-08-11T05:09:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3065 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:09:01 dist-test-slave-7c35 krb5kdc[8338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:09:01 dist-test-slave-7c35 krb5kdc[8338](info): set up 2 sockets
Aug 11 05:09:01 dist-test-slave-7c35 krb5kdc[8338](info): commencing operation
krb5kdc: starting...
W20260811 05:09:02.197016 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:09:02 dist-test-slave-7c35 krb5kdc[8338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:09:02.315972 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35735
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:02.422019 8352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:02.422247 8352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:02.422283 8352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:02.426009 8352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:02.426056 8352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:02.426075 8352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:02.426091 8352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:02.426146 8352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:02.430886 8352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:02.431787 8352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:02.432755 8352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:02.440703 8358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:02.440874 8352 server_base.cc:1034] running on GCE node
W20260811 05:09:02.440694 8357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:02.440694 8362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:02.441623 8352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:02.442544 8352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:02.443706 8352 hybrid_clock.cc:648] HybridClock initialized: now 1786424942443681 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T05:09:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1335 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:09:02 dist-test-slave-7c35 krb5kdc[8367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:09:02 dist-test-slave-7c35 krb5kdc[8367](info): set up 2 sockets
Aug 11 05:09:02 dist-test-slave-7c35 krb5kdc[8367](info): commencing operation
krb5kdc: starting...
W20260811 05:09:03.528789 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:09:03 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:09:03.653986 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41699
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:41699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:03.761674 8380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:03.761920 8380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:03.761988 8380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:03.765791 8380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:03.765874 8380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:03.765911 8380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:03.765964 8380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:03.766004 8380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:03.770682 8380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:41699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:03.771662 8380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:03.772565 8380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:03.780478 8388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:03.780485 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:03.780488 8390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:03.780696 8380 server_base.cc:1034] running on GCE node
I20260811 05:09:03.781270 8380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:03.782226 8380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:03.783396 8380 hybrid_clock.cc:648] HybridClock initialized: now 1786424943783371 us; error 37 us; skew 500 ppm
Aug 11 05:09:03 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424943, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:03.790458 8380 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:09:03.791317 8380 webserver.cc:460] Webserver started at http://127.30.88.254:33769/ using document root <none> and password file <none>
I20260811 05:09:03.791859 8380 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:03.791944 8380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:03.792160 8380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:03.794104 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "b3d8d1d5378a4d6d93bcc536563bba36"
format_stamp: "Formatted at 2026-08-11 05:09:03 on dist-test-slave-7c35"
server_key: "793d10fc2ad9066308b4e076500fda2e"
server_key_iv: "7920ef34b3a5c1b76238b13daf18e7e5"
server_key_version: "encryptionkey@0"
I20260811 05:09:03.794641 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "b3d8d1d5378a4d6d93bcc536563bba36"
format_stamp: "Formatted at 2026-08-11 05:09:03 on dist-test-slave-7c35"
server_key: "793d10fc2ad9066308b4e076500fda2e"
server_key_iv: "7920ef34b3a5c1b76238b13daf18e7e5"
server_key_version: "encryptionkey@0"
I20260811 05:09:03.798208 8380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:09:03.800496 8398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:03.801606 8380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:09:03.801740 8380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "b3d8d1d5378a4d6d93bcc536563bba36"
format_stamp: "Formatted at 2026-08-11 05:09:03 on dist-test-slave-7c35"
server_key: "793d10fc2ad9066308b4e076500fda2e"
server_key_iv: "7920ef34b3a5c1b76238b13daf18e7e5"
server_key_version: "encryptionkey@0"
I20260811 05:09:03.801843 8380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:03.815586 8380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:03.818513 8380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:03.818702 8380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:03.825839 8380 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:41699
I20260811 05:09:03.825877 8455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:41699 every 8 connection(s)
I20260811 05:09:03.826758 8380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:09:03.829425 8456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:03.830030 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8380
I20260811 05:09:03.830199 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:09:03.830538 31075 external_mini_cluster.cc:1428] Setting key 53173ad600f32c49229eca5c7a25f004
I20260811 05:09:03.836150 8456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Bootstrap starting.
I20260811 05:09:03.838419 8456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Neither blocks nor log segments found. Creating new log.
Aug 11 05:09:03 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:03.839341 8456 log.cc:824] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:03.841671 8456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: No bootstrap required, opened a new log
I20260811 05:09:03.844956 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:03.831976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.30.88.254:41699)
0811 05:09:03.832396 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:03.832410 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:03.832468 (+ 58us) server_negotiation.cc:407] Connection header received
0811 05:09:03.833145 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:03.833162 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:03.833463 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:03.833792 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:03.834599 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:03.835667 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:03.836577 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:03.837039 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:03.837203 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:03.839984 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:03.839989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:03.840001 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:03.840026 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:03.841930 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:03.842350 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:03.842356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:03.842362 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:03.842507 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:03.842778 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:03.842781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:03.842783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:03.843286 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:03.843500 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:03.843712 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 05:09:03.844059 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260811 05:09:03.846151 8456 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:03.846374 8456 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:03.846457 8456 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d8d1d5378a4d6d93bcc536563bba36, State: Initialized, Role: FOLLOWER
I20260811 05:09:03.846956 8456 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:03.847127 8456 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:09:03.847198 8456 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:09:03.847293 8456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:03.848160 8456 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:03.848497 8456 leader_election.cc:302] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3d8d1d5378a4d6d93bcc536563bba36; no voters:
I20260811 05:09:03.848793 8456 leader_election.cc:288] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:09:03.848919 8462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:03.849150 8462 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 LEADER]: Becoming Leader. State: Replica: b3d8d1d5378a4d6d93bcc536563bba36, State: Running, Role: LEADER
I20260811 05:09:03.849467 8462 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:03.849996 8456 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:09:03.851207 8464 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d8d1d5378a4d6d93bcc536563bba36. Latest consensus state: current_term: 1 leader_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } } }
I20260811 05:09:03.851347 8464 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:03.851622 8463 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } } }
I20260811 05:09:03.851805 8463 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:03.851713 8473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:09:03.855274 8473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:09:03.859993 8473 catalog_manager.cc:1347] Generated new cluster ID: fc0a1c7eb79c4f54a886ebe0ef3e6867
I20260811 05:09:03.860054 8473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:09:03.873816 8473 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:09:03.874754 8473 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:09:03.882568 8473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Generated new TSK 0
I20260811 05:09:03.883280 8473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:09:03.915848 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:04.021181 8490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:04.021418 8490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:04.021456 8490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:04.025113 8490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:04.025158 8490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:04.025231 8490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:09:04.029857 8490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:04.030803 8490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:04.031747 8490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:04.040354 8495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.040374 8498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.040365 8496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.040647 8490 server_base.cc:1034] running on GCE node
I20260811 05:09:04.041137 8490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:04.041657 8490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:04.042814 8490 hybrid_clock.cc:648] HybridClock initialized: now 1786424944042792 us; error 32 us; skew 500 ppm
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:04.045554 8490 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:09:04.046380 8490 webserver.cc:460] Webserver started at http://127.30.88.193:41311/ using document root <none> and password file <none>
I20260811 05:09:04.046988 8490 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:04.047070 8490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:04.047297 8490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:04.049098 8490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "13e5ea29eb0e446da5c13fdf9a5f06e6"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "b22133690b7eb18166395753cdf9d426"
server_key_iv: "5bf4dd6fa39fa35f893a53715d1fe32d"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.049615 8490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "13e5ea29eb0e446da5c13fdf9a5f06e6"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "b22133690b7eb18166395753cdf9d426"
server_key_iv: "5bf4dd6fa39fa35f893a53715d1fe32d"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.053205 8490 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 05:09:04.055371 8506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.056494 8490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:04.056622 8490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "13e5ea29eb0e446da5c13fdf9a5f06e6"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "b22133690b7eb18166395753cdf9d426"
server_key_iv: "5bf4dd6fa39fa35f893a53715d1fe32d"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.056722 8490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:04.069748 8490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:04.072539 8490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:04.072743 8490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:04.073216 8490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:04.073916 8490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:04.073978 8490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.074036 8490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:04.074077 8490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.081944 8490 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:43065
I20260811 05:09:04.081966 8634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:43065 every 8 connection(s)
I20260811 05:09:04.082775 8490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:09:04.091189 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8490
I20260811 05:09:04.091331 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:04.091614 31075 external_mini_cluster.cc:1428] Setting key 980b194321549bab4c137d79e7d3fe0c
I20260811 05:09:04.095332 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.084490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:56929 (local address 127.30.88.254:41699)
0811 05:09:04.084622 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:04.084627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:04.085385 (+ 758us) server_negotiation.cc:407] Connection header received
0811 05:09:04.086623 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:04.086625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:04.086673 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:04.086748 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:04.087779 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.088278 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.089312 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.089516 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.089592 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.091879 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:04.091882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:04.091885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:04.091910 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:04.093477 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.093898 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.093904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.093908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.094001 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.094289 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.094294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.094298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.094532 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:04.094630 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:04.095063 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 05:09:04.095175 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:09:04.096056 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.084823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:56929)
0811 05:09:04.085244 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:04.085282 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:04.086404 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:04.086949 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:04.086955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:04.087253 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:04.087659 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.087670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.088373 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.088375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.089194 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.089203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.089629 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.089632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.089754 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.090294 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:04.090310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:04.091730 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:04.093582 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.093586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.093595 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.093808 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.094088 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.094090 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.094091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.094208 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.094626 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:04.094630 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:04.094808 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 05:09:04.095015 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 05:09:04.095283 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260811 05:09:04.097124 8635 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41699
I20260811 05:09:04.097342 8635 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:04.097885 8635 heartbeater.cc:511] Master 127.30.88.254:41699 requested a full tablet report, sending...
I20260811 05:09:04.099655 8417 ts_manager.cc:194] Registered new tserver with Master: 13e5ea29eb0e446da5c13fdf9a5f06e6 (127.30.88.193:43065)
I20260811 05:09:04.101315 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:56929
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:09:04.120014 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:04.233804 8640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:04.234076 8640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:04.234165 8640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:04.238391 8640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:04.238476 8640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:04.238566 8640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:09:04.243680 8640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:04.244715 8640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:04.245731 8640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:04.255502 8648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.255515 8647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.255502 8650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.255887 8640 server_base.cc:1034] running on GCE node
I20260811 05:09:04.256279 8640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:04.256906 8640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:04.258111 8640 hybrid_clock.cc:648] HybridClock initialized: now 1786424944258094 us; error 34 us; skew 500 ppm
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:04.260879 8640 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:09:04.261672 8640 webserver.cc:460] Webserver started at http://127.30.88.194:42875/ using document root <none> and password file <none>
I20260811 05:09:04.262207 8640 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:04.262262 8640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:04.262418 8640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:04.264394 8640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "90d70d518a92473bbdcd035cdf10513e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "c65a32203d5152ff0616f5b74d1f0fa8"
server_key_iv: "47a7e0c54f616d33ecee442fec3182af"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.264967 8640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "90d70d518a92473bbdcd035cdf10513e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "c65a32203d5152ff0616f5b74d1f0fa8"
server_key_iv: "47a7e0c54f616d33ecee442fec3182af"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.268673 8640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 05:09:04.271179 8657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.272382 8640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:09:04.272508 8640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "90d70d518a92473bbdcd035cdf10513e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "c65a32203d5152ff0616f5b74d1f0fa8"
server_key_iv: "47a7e0c54f616d33ecee442fec3182af"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.272614 8640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:04.287672 8640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:04.290833 8640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:04.291078 8640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:04.291666 8640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:04.292456 8640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:04.292505 8640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.292572 8640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:04.292609 8640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.301157 8640 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:42179
I20260811 05:09:04.301241 8791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:42179 every 8 connection(s)
I20260811 05:09:04.301999 8640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:09:04.306357 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8640
I20260811 05:09:04.306507 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:09:04.306885 31075 external_mini_cluster.cc:1428] Setting key ec70180a177b78d52c3cdf9d67352582
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:04.316625 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.303798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:52817 (local address 127.30.88.254:41699)
0811 05:09:04.303917 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:04.303922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:04.304642 (+ 720us) server_negotiation.cc:407] Connection header received
0811 05:09:04.305956 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:04.305959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:04.306010 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:04.306086 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:04.307574 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.308084 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.309785 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.310018 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.310113 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.312975 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:04.312978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:04.312980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:04.313004 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:04.314378 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.314948 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.314953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.314958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.315061 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.315387 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.315401 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.315406 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.315673 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:04.315822 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:04.316288 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 05:09:04.316445 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:09:04.317206 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.304087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:52817)
0811 05:09:04.304501 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:04.304538 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:04.305764 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:04.306285 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:04.306296 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:04.306787 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:04.307423 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.307439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.308207 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.308211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.309653 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.309668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.310178 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.310185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.310414 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.311080 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:04.311094 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:04.312794 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:04.314478 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.314483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.314494 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.314796 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.315156 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.315159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.315160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.315281 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.315822 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:04.315826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:04.316034 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 05:09:04.316172 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 05:09:04.316397 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:09:04.318312 8792 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41699
I20260811 05:09:04.318511 8792 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:04.319005 8792 heartbeater.cc:511] Master 127.30.88.254:41699 requested a full tablet report, sending...
I20260811 05:09:04.320012 8417 ts_manager.cc:194] Registered new tserver with Master: 90d70d518a92473bbdcd035cdf10513e (127.30.88.194:42179)
I20260811 05:09:04.320597 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:52817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:09:04.337244 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:04.443579 8798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:04.443784 8798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:04.443823 8798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:04.447589 8798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:04.447630 8798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:04.447682 8798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:04.452602 8798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:04.453413 8798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:04.454286 8798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:04.463030 8804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.463254 8798 server_base.cc:1034] running on GCE node
W20260811 05:09:04.463027 8807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.463030 8805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.463730 8798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:04.464222 8798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:04.465372 8798 hybrid_clock.cc:648] HybridClock initialized: now 1786424944465356 us; error 46 us; skew 500 ppm
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:04.468017 8798 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:09:04.468811 8798 webserver.cc:460] Webserver started at http://127.30.88.195:35635/ using document root <none> and password file <none>
I20260811 05:09:04.469297 8798 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:04.469376 8798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:04.469575 8798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:04.471477 8798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "98aba57f4c5a42c59717d67d4433780e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "76ab218dccc0d082eee66098988f2111"
server_key_iv: "91ac88984683b80c620fd56655984bc5"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.472009 8798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "98aba57f4c5a42c59717d67d4433780e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "76ab218dccc0d082eee66098988f2111"
server_key_iv: "91ac88984683b80c620fd56655984bc5"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.475423 8798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:09:04.477684 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.478654 8798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:04.478768 8798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "98aba57f4c5a42c59717d67d4433780e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "76ab218dccc0d082eee66098988f2111"
server_key_iv: "91ac88984683b80c620fd56655984bc5"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.478863 8798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:04.507500 8798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:04.510040 8798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:04.510253 8798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:04.510789 8798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:04.511580 8798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:04.511644 8798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.511708 8798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:04.511754 8798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.520083 8798 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:45115
I20260811 05:09:04.520112 8943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:45115 every 8 connection(s)
I20260811 05:09:04.520877 8798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:04.523272 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8798
I20260811 05:09:04.523396 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:09:04.523674 31075 external_mini_cluster.cc:1428] Setting key 5c810ba7e6eafaa8c4cc4ab2b2a50b3b
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:04.534701 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.522597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59431 (local address 127.30.88.254:41699)
0811 05:09:04.522735 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:04.522739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:04.523559 (+ 820us) server_negotiation.cc:407] Connection header received
0811 05:09:04.525104 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:04.525107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:04.525150 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:04.525213 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:04.526505 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.527273 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.528382 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.528625 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.528741 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.530903 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:04.530907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:04.530910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:04.530934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:04.532401 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.532896 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.532899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.532901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.533024 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:04.533555 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:04.533560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:04.533564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:04.533812 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:04.533919 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:04.534414 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 05:09:04.534536 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:09:04.535483 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:04.522978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:59431)
0811 05:09:04.523409 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:04.523457 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:04.524898 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:04.525366 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:04.525374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:04.525814 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:04.526338 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.526356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.527391 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.527394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.528256 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:04.528265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:04.528776 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:04.528780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:04.528948 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:04.529396 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:04.529410 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:04.530734 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:04.532538 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.532543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.532555 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.532800 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.533144 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:04.533148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:04.533150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:04.533345 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:04.533965 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:04.533971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:04.534182 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 05:09:04.534357 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 05:09:04.534644 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":117,"threads_started":1}
I20260811 05:09:04.536634 8944 heartbeater.cc:347] Connected to a master server at 127.30.88.254:41699
I20260811 05:09:04.536878 8944 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:04.537382 8944 heartbeater.cc:511] Master 127.30.88.254:41699 requested a full tablet report, sending...
I20260811 05:09:04.538337 8417 ts_manager.cc:194] Registered new tserver with Master: 98aba57f4c5a42c59717d67d4433780e (127.30.88.195:45115)
I20260811 05:09:04.538949 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:59431
I20260811 05:09:04.548610 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
Entry for principal oryx/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab.
WARNING: no policy specified for oryx/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
Entry for principal oryx/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab.
WARNING: no policy specified for oryx/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
Entry for principal oryx/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab.
I20260811 05:09:04.622303 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8490
I20260811 05:09:04.632931 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8640
I20260811 05:09:04.639671 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8798
I20260811 05:09:04.647924 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8380
I20260811 05:09:04.656674 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41699
--webserver_interface=127.30.88.254
--webserver_port=33769
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:41699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:04.766005 8955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:04.766258 8955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:04.766343 8955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:04.769984 8955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:04.770035 8955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:04.770054 8955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:04.770153 8955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:04.770195 8955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:04.775182 8955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:41699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:04.776201 8955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:04.777217 8955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:04.785559 8961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.785559 8960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.785558 8963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.785797 8955 server_base.cc:1034] running on GCE node
I20260811 05:09:04.786404 8955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:04.787537 8955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:04.788748 8955 hybrid_clock.cc:648] HybridClock initialized: now 1786424944788720 us; error 39 us; skew 500 ppm
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:04.791617 8955 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:09:04.792527 8955 webserver.cc:460] Webserver started at http://127.30.88.254:33769/ using document root <none> and password file <none>
I20260811 05:09:04.793097 8955 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:04.793217 8955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:04.797039 8955 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 05:09:04.799290 8973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.800566 8955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:04.800714 8955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "b3d8d1d5378a4d6d93bcc536563bba36"
format_stamp: "Formatted at 2026-08-11 05:09:03 on dist-test-slave-7c35"
server_key: "793d10fc2ad9066308b4e076500fda2e"
server_key_iv: "7920ef34b3a5c1b76238b13daf18e7e5"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.801249 8955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:04.820592 8955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:04.823921 8955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:04.824162 8955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:04.831595 8955 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:41699
I20260811 05:09:04.831637 9033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:41699 every 8 connection(s)
I20260811 05:09:04.832644 8955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:09:04.836925 9034 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:09:04.838615 9034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Bootstrap starting.
I20260811 05:09:04.842895 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 8955
I20260811 05:09:04.843326 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:43065
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:41699
--webserver_port=41311
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:09:04.855284 9034 log.cc:824] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:04.861991 9034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:09:04.862390 9034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Bootstrap complete.
I20260811 05:09:04.866279 9034 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:04.866727 9034 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d8d1d5378a4d6d93bcc536563bba36, State: Initialized, Role: FOLLOWER
I20260811 05:09:04.867249 9034 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:04.867416 9034 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:09:04.867498 9034 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:09:04.867612 9034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:09:04.869465 9034 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:04.869774 9034 leader_election.cc:302] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3d8d1d5378a4d6d93bcc536563bba36; no voters:
I20260811 05:09:04.870100 9034 leader_election.cc:288] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:09:04.870200 9038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:09:04.870394 9038 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [term 2 LEADER]: Becoming Leader. State: Replica: b3d8d1d5378a4d6d93bcc536563bba36, State: Running, Role: LEADER
I20260811 05:09:04.870677 9038 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } }
I20260811 05:09:04.871395 9034 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:09:04.872015 9039 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } } }
I20260811 05:09:04.872357 9039 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:04.872269 9040 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3d8d1d5378a4d6d93bcc536563bba36. Latest consensus state: current_term: 2 leader_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3d8d1d5378a4d6d93bcc536563bba36" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41699 } } }
I20260811 05:09:04.872730 9040 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:04.873034 9047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:09:04.876410 9047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:09:04.877295 9047 catalog_manager.cc:1259] Loaded cluster ID: fc0a1c7eb79c4f54a886ebe0ef3e6867
I20260811 05:09:04.877354 9047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:09:04.879639 9047 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:09:04.880241 9047 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3d8d1d5378a4d6d93bcc536563bba36: Loaded TSK: 0
I20260811 05:09:04.881395 9047 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:09:04.952852 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:04.953059 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:04.953094 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:04.956751 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:04.956800 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:04.956854 9036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:09:04.961508 9036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.193.keytab
--principal=oryx/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=41311
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:04.962378 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:04.963301 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:04.973147 9065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.973176 9062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:04.973176 9063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:04.974020 9036 server_base.cc:1034] running on GCE node
I20260811 05:09:04.974423 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:04.975088 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:04.976265 9036 hybrid_clock.cc:648] HybridClock initialized: now 1786424944976220 us; error 63 us; skew 500 ppm
Aug 11 05:09:04 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424944, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:04.979146 9036 init.cc:377] Logged in from keytab as oryx/127.30.88.193@KRBTEST.COM (short username oryx)
I20260811 05:09:04.979979 9036 webserver.cc:460] Webserver started at http://127.30.88.193:41311/ using document root <none> and password file <none>
I20260811 05:09:04.980487 9036 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:04.980551 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:04.984112 9036 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:09:04.986264 9075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:04.987536 9036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:04.987650 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "13e5ea29eb0e446da5c13fdf9a5f06e6"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "b22133690b7eb18166395753cdf9d426"
server_key_iv: "5bf4dd6fa39fa35f893a53715d1fe32d"
server_key_version: "encryptionkey@0"
I20260811 05:09:04.988121 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:04.998174 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:05.001753 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:05.002045 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:05.002574 9036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:05.003441 9036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:05.003506 9036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.003571 9036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:05.003616 9036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.013849 9036 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:43065
I20260811 05:09:05.013876 9209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:43065 every 8 connection(s)
I20260811 05:09:05.014791 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:09:05.020140 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9036
I20260811 05:09:05.020581 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:42179
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:41699
--webserver_port=42875
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:09:05.027484 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.016695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:39505 (local address 127.30.88.254:41699)
0811 05:09:05.017115 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.017130 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.017303 (+ 173us) server_negotiation.cc:407] Connection header received
0811 05:09:05.018743 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.018753 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.019091 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.019347 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.020432 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.021401 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.022467 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.022958 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.023125 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.026761 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:39505: BlockingRecv error: recv got EOF from 127.30.88.193:39505 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":129,"threads_started":1}
I20260811 05:09:05.027503 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.016708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:39505)
0811 05:09:05.017150 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.017197 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.018418 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.019440 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.019446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.019792 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.020243 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.020278 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.021548 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.021551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.022339 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.022348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.023122 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.023126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.023301 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.023854 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.023870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.026558 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":130,"threads_started":1}
W20260811 05:09:05.028056 9210 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:41699 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.88.254:41699: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.034231 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.028455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:34541)
0811 05:09:05.028602 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.028617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.028698 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.028933 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.028937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.029106 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.029257 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.029268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.029938 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.029945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.031037 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.031051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.031511 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.031517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.031645 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.032146 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.032165 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.034101 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 05:09:05.034332 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.028564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:34541 (local address 127.30.88.254:41699)
0811 05:09:05.028669 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.028672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.028684 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:09:05.028782 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.028783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.028816 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.028882 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.029379 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.029822 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.031162 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.031379 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.031477 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.034224 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:34541: BlockingRecv error: recv got EOF from 127.30.88.193:34541 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.039546 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.034732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:33427)
0811 05:09:05.034832 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.034865 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.034934 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.035120 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.035123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.035274 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.035363 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.035368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.036056 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.036059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.036900 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.036909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.037314 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.037317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.037405 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.037760 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.037774 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.039443 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 05:09:05.039635 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.034823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:33427 (local address 127.30.88.254:41699)
0811 05:09:05.034940 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.034943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.034954 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:09:05.034995 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.034996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.035027 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.035087 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.035480 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.035952 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.037034 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.037207 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.037290 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.039539 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:33427: BlockingRecv error: recv got EOF from 127.30.88.193:33427 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 05:09:05.039702 9210 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:09:05.131994 9215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:05.132215 9215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:05.132253 9215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:05.136078 9215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:05.136128 9215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:05.136183 9215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:09:05.140909 9215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.194.keytab
--principal=oryx/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:42179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=42875
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:05.141806 9215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:05.142831 9215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:05.152066 9224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:05.152086 9221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:05.152086 9222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:05.152135 9215 server_base.cc:1034] running on GCE node
I20260811 05:09:05.152591 9215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:05.153155 9215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:05.154302 9215 hybrid_clock.cc:648] HybridClock initialized: now 1786424945154286 us; error 27 us; skew 500 ppm
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424945, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:05.156957 9215 init.cc:377] Logged in from keytab as oryx/127.30.88.194@KRBTEST.COM (short username oryx)
I20260811 05:09:05.157752 9215 webserver.cc:460] Webserver started at http://127.30.88.194:42875/ using document root <none> and password file <none>
I20260811 05:09:05.158233 9215 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:05.158303 9215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:05.161675 9215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:05.163350 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.164408 9215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:05.164534 9215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "90d70d518a92473bbdcd035cdf10513e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "c65a32203d5152ff0616f5b74d1f0fa8"
server_key_iv: "47a7e0c54f616d33ecee442fec3182af"
server_key_version: "encryptionkey@0"
I20260811 05:09:05.164974 9215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:05.185508 9215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:05.188675 9215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:05.188859 9215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:05.189446 9215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:05.190258 9215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:05.190304 9215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.190369 9215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:05.190403 9215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.199429 9215 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:42179
I20260811 05:09:05.199463 9359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:42179 every 8 connection(s)
I20260811 05:09:05.200366 9215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:09:05.207561 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9215
I20260811 05:09:05.207973 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:45115
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:41699
--webserver_port=35635
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:09:05.211217 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.202096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:34127 (local address 127.30.88.254:41699)
0811 05:09:05.202219 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.202223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.203104 (+ 881us) server_negotiation.cc:407] Connection header received
0811 05:09:05.204592 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.204595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.204668 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.204763 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.205951 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.206472 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.207636 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.207893 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.207999 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.211095 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:34127: BlockingRecv error: recv got EOF from 127.30.88.194:34127 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:09:05.211789 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.202419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:34127)
0811 05:09:05.202943 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.202992 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.204385 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.204936 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.204943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.205310 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.205795 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.205809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.206593 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.206596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.207505 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.207529 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.208038 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.208041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.208197 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.208730 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.208745 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.210986 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":118,"threads_started":1}
W20260811 05:09:05.212280 9360 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:41699 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.88.254:41699: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.218135 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.212743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:44897)
0811 05:09:05.212843 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.212866 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.212935 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.213184 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.213186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.213304 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.213390 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.213394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.214406 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.214408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.214961 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.214969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.215593 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.215596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.215664 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.216136 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.216147 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.217990 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:09:05.218323 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.212825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:44897 (local address 127.30.88.254:41699)
0811 05:09:05.212958 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.212964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.212980 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:09:05.213024 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.213028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.213076 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.213175 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.213496 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.214294 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.215068 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.215459 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.215585 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.218175 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:44897: BlockingRecv error: recv got EOF from 127.30.88.194:44897 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.224841 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.219690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:38911)
0811 05:09:05.219938 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.219956 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.220053 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.220282 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.220285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.220473 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.220567 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.220574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.221243 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.221247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.222083 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.222093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.222479 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.222483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.222566 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.223002 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.223017 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.224731 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:09:05.225020 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.219738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:38911 (local address 127.30.88.254:41699)
0811 05:09:05.219856 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.219861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.220024 (+ 163us) server_negotiation.cc:407] Connection header received
0811 05:09:05.220139 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.220141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.220174 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.220237 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.220695 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.221145 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.222188 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.222383 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.222454 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.224896 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:38911: BlockingRecv error: recv got EOF from 127.30.88.194:38911 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 05:09:05.225174 9360 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:09:05.319236 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:05.319471 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:05.319553 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:05.323462 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:05.323525 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:05.323621 9363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:05.328646 9363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/krb5kdc/oryx_127.30.88.195.keytab
--principal=oryx/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:45115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=35635
--tserver_master_addrs=127.30.88.254:41699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:05.329531 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:05.330474 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:05.339841 9370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:05.339911 9373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:05.339912 9371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:05.340272 9363 server_base.cc:1034] running on GCE node
I20260811 05:09:05.340667 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:05.341233 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:05.342406 9363 hybrid_clock.cc:648] HybridClock initialized: now 1786424945342379 us; error 44 us; skew 500 ppm
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424945, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:05.345033 9363 init.cc:377] Logged in from keytab as oryx/127.30.88.195@KRBTEST.COM (short username oryx)
I20260811 05:09:05.345892 9363 webserver.cc:460] Webserver started at http://127.30.88.195:35635/ using document root <none> and password file <none>
I20260811 05:09:05.346431 9363 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:05.346537 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:05.349996 9363 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:09:05.351900 9380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.352960 9363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:09:05.353077 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "98aba57f4c5a42c59717d67d4433780e"
format_stamp: "Formatted at 2026-08-11 05:09:04 on dist-test-slave-7c35"
server_key: "76ab218dccc0d082eee66098988f2111"
server_key_iv: "91ac88984683b80c620fd56655984bc5"
server_key_version: "encryptionkey@0"
I20260811 05:09:05.353549 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:05.363749 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:05.366212 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:05.366392 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:05.366955 9363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:05.367714 9363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:05.367774 9363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.367830 9363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:05.367872 9363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:05.377321 9363 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:45115
I20260811 05:09:05.377341 9504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:45115 every 8 connection(s)
I20260811 05:09:05.378274 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:05.385226 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9363
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:09:05.389257 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.379952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:34527 (local address 127.30.88.254:41699)
0811 05:09:05.380077 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.380080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.380843 (+ 763us) server_negotiation.cc:407] Connection header received
0811 05:09:05.382212 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.382216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.382278 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.382374 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.383527 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.384043 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.385223 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.385522 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.385636 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.389133 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:34527: BlockingRecv error: recv got EOF from 127.30.88.195:34527 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:09:05.389935 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.380258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:34527)
0811 05:09:05.380709 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.380748 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.381992 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.382542 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.382549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.382928 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.383379 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.383393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.384167 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.384171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.385103 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.385113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.385673 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.385677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.385858 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.386483 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.386503 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.389015 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":170,"threads_started":1}
W20260811 05:09:05.390417 9505 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:41699 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.88.254:41699: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.395128 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.386279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.30.88.254:41699)
0811 05:09:05.386578 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.386583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.386630 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:09:05.386755 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.386759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.386807 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.386958 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.387635 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.388278 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.389109 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.389419 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.389513 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.391178 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:05.391183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:05.391196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:05.391223 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:05.393014 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:05.393471 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:05.393482 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:05.393487 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:05.393611 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:05.393888 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:05.393891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:05.393893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:05.394201 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:05.394408 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:05.394640 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 05:09:05.395011 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.395819 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.390803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:37027)
0811 05:09:05.390916 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.390930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.391010 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.391288 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.391291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.391452 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.391583 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.391591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.392577 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.392579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.393076 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.393084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.393492 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.393495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.393588 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.393995 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.394012 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.395716 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 05:09:05.395965 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.390907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:37027 (local address 127.30.88.254:41699)
0811 05:09:05.391014 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.391018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.391032 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:05.391150 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.391153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.391194 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.391273 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.391712 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.392469 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.393191 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.393391 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.393499 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.395820 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:37027: BlockingRecv error: recv got EOF from 127.30.88.195:37027 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:05 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:05.400761 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:05.396306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:54143)
0811 05:09:05.396391 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:05.396403 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:05.396461 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:05.396805 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:05.396808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:05.396955 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:05.397023 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.397066 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.397895 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.397898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.398420 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:05.398429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.398792 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.398794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:05.398918 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.399254 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:05.399264 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:05.400665 (+ 1401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 05:09:05.400923 9213 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:05.396520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:54143 (local address 127.30.88.254:41699)
0811 05:09:05.396632 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:05.396635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:05.396644 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:09:05.396669 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:05.396671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:05.396699 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:05.396755 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:05.397186 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.397773 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.398517 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:05.398704 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:05.398776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:05.400785 (+ 2009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:54143: BlockingRecv error: recv got EOF from 127.30.88.195:54143 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 05:09:05.400969 9505 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:06.045755 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:06.040340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:48901)
0811 05:09:06.040627 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:06.040641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:06.040738 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:06.040994 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:06.040997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:06.041184 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:06.041287 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.041292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.042009 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.042011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.042584 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.042592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.043064 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.043067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.043130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.043653 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:06.043666 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:06.045630 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
W20260811 05:09:06.045902 9511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:06.040429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:48901 (local address 127.30.88.254:41699)
0811 05:09:06.040707 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:06.040711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:06.040726 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:09:06.040827 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:06.040829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:06.040875 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:06.040957 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:06.041388 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.041916 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.042684 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.042953 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.043035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.045767 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:48901: BlockingRecv error: recv got EOF from 127.30.88.193:48901 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:06.231269 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:06.225778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:37185)
0811 05:09:06.226058 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:06.226076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:06.226206 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:06.226468 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:06.226483 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:06.226732 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:06.226940 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.226952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.227693 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.227698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.228253 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.228261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.228707 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.228711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.228803 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.229276 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:06.229287 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:06.231158 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
W20260811 05:09:06.231606 9511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:06.225855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:37185 (local address 127.30.88.254:41699)
0811 05:09:06.226027 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:06.226033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:06.226148 (+ 115us) server_negotiation.cc:407] Connection header received
0811 05:09:06.226310 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:06.226314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:06.226362 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:06.226441 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:06.227064 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.227589 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.228368 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.228592 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.228691 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.231343 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:37185: BlockingRecv error: recv got EOF from 127.30.88.194:37185 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:06 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:06.407251 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:06.401521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:55911)
0811 05:09:06.401784 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:06.401797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:06.401942 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:06.402213 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:06.402216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:06.402438 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:06.402537 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.402543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.403368 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.403374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.403949 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:06.403957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.404472 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.404476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:06.404572 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.405175 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:06.405188 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:06.407065 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
W20260811 05:09:06.407315 9511 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:06.401575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:55911 (local address 127.30.88.254:41699)
0811 05:09:06.401731 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:06.401735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:06.401867 (+ 132us) server_negotiation.cc:407] Connection header received
0811 05:09:06.402047 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:06.402050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:06.402105 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:06.402186 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:06.402654 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.403254 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.404089 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:06.404358 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:06.404468 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:06.407205 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:55911: BlockingRecv error: recv got EOF from 127.30.88.195:55911 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:07.052134 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:07.046667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:40905)
0811 05:09:07.046999 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:07.047017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:07.047125 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:07.047414 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:07.047416 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:07.047609 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:07.047719 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.047725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.048450 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.048452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.049010 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.049018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.049419 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.049422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.049486 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.049988 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:07.050001 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:07.051933 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
W20260811 05:09:07.052193 9515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:07.046745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:40905 (local address 127.30.88.254:41699)
0811 05:09:07.047077 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:07.047081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:07.047095 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:07.047218 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:07.047220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:07.047266 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:07.047345 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:07.047820 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.048354 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.049107 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.049322 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.049396 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.052065 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:40905: BlockingRecv error: recv got EOF from 127.30.88.193:40905 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":160,"threads_started":1}
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:07.237052 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:07.232131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:56527)
0811 05:09:07.232393 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:07.232410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:07.232507 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:07.232737 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:07.232739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:07.232927 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:07.233021 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.233026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.233727 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.233729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.234264 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.234272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.234660 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.234663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.234727 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.235209 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:07.235220 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:07.236936 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
W20260811 05:09:07.237243 9515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:07.232206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:56527 (local address 127.30.88.254:41699)
0811 05:09:07.232334 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:07.232337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:07.232483 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:09:07.232583 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:07.232585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:07.232629 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:07.232700 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:07.233114 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.233638 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.234356 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.234565 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.234635 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.237102 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:56527: BlockingRecv error: recv got EOF from 127.30.88.194:56527 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:07 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:07.413524 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:07.408105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:40207)
0811 05:09:07.408435 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:07.408448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:07.408585 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:07.408855 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:07.408858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:07.409069 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:07.409166 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.409172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.409942 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.409945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.410513 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:07.410521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.411014 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.411018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:07.411106 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.411579 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:07.411591 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:07.413363 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
W20260811 05:09:07.413599 9515 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:07.408224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:40207 (local address 127.30.88.254:41699)
0811 05:09:07.408344 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:07.408348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:07.408526 (+ 178us) server_negotiation.cc:407] Connection header received
0811 05:09:07.408702 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:07.408704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:07.408751 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:07.408832 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:07.409289 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.409817 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.410635 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:07.410903 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:07.411010 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:07.413503 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:40207: BlockingRecv error: recv got EOF from 127.30.88.195:40207 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:08.058736 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:08.052955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:39105)
0811 05:09:08.053247 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:08.053260 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:08.053355 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:08.053582 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:08.053584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:08.053770 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:08.053881 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.053887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.054656 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.054660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.055251 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.055261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.055699 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.055702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.055768 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.056365 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:08.056377 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:08.058543 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
W20260811 05:09:08.058796 9519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:08.053137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:39105 (local address 127.30.88.254:41699)
0811 05:09:08.053362 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:08.053366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:08.053382 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:09:08.053423 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:08.053425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:08.053464 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:08.053545 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:08.053982 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.054526 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.055361 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.055596 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.055678 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.058666 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:39105: BlockingRecv error: recv got EOF from 127.30.88.193:39105 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:08.243369 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:08.237763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:38243)
0811 05:09:08.238075 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:08.238092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:08.238237 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:08.238500 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:08.238504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:08.238714 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:08.238806 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.238812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.239645 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.239648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.240364 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.240374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.240907 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.240911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.241002 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.241472 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:08.241484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:08.243245 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
W20260811 05:09:08.243492 9519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:08.237945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:38243 (local address 127.30.88.254:41699)
0811 05:09:08.238086 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:08.238092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:08.238177 (+ 85us) server_negotiation.cc:407] Connection header received
0811 05:09:08.238333 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:08.238337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:08.238388 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:08.238484 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:08.238968 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.239514 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.240497 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.240782 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.240889 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.243378 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:38243: BlockingRecv error: recv got EOF from 127.30.88.194:38243 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:08 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:08.419653 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:08.414345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:36961)
0811 05:09:08.414619 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:08.414632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:08.414769 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:08.415053 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:08.415056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:08.415267 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:08.415362 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.415368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.416119 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.416126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.416678 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:08.416686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.417138 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.417142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:08.417228 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.417700 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:08.417711 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:08.419488 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
W20260811 05:09:08.419715 9519 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:08.414403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:36961 (local address 127.30.88.254:41699)
0811 05:09:08.414558 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:08.414564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:08.414710 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:09:08.414899 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:08.414902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:08.414950 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:08.415031 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:08.415486 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.416016 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.416797 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:08.417020 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:08.417124 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:08.419607 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:36961: BlockingRecv error: recv got EOF from 127.30.88.195:36961 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:09.069988 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:09.059563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:38801)
0811 05:09:09.059879 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:09.059894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:09.059991 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:09.060289 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:09.060291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:09.060525 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:09.060642 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.060648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.061412 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.061415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.062030 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.062040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.062503 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.062510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.062604 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.067872 (+ 5268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:09.067886 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:09.069851 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
W20260811 05:09:09.070134 9523 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:09.059695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38801 (local address 127.30.88.254:41699)
0811 05:09:09.059939 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:09.059942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:09.059956 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:09.060101 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:09.060103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:09.060160 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:09.060239 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:09.060748 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.061280 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.062142 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.062385 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.062460 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.069988 (+ 7528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:38801: BlockingRecv error: recv got EOF from 127.30.88.193:38801 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:09.249722 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:09.244129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:35107)
0811 05:09:09.244410 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:09.244424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:09.244534 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:09.244768 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:09.244770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:09.244994 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:09.245138 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.245145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.246101 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.246105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.246672 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.246681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.247180 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.247183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.247248 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.247758 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:09.247769 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:09.249553 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
W20260811 05:09:09.249768 9523 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:09.244342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:35107 (local address 127.30.88.254:41699)
0811 05:09:09.244523 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:09.244527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:09.244542 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:09:09.244613 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:09.244615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:09.244656 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:09.244725 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:09.245277 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.246003 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.246774 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.247074 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.247156 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.249666 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:35107: BlockingRecv error: recv got EOF from 127.30.88.194:35107 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:09 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:09.425738 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:09.420551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:56875)
0811 05:09:09.420815 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:09.420827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:09.420920 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:09.421145 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:09.421147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:09.421319 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:09.421419 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.421424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.422110 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.422112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.422680 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:09.422687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.423140 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.423143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:09.423207 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.423676 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:09.423687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:09.425567 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
W20260811 05:09:09.425783 9523 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:09.420613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:56875 (local address 127.30.88.254:41699)
0811 05:09:09.420730 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:09.420733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:09.420895 (+ 162us) server_negotiation.cc:407] Connection header received
0811 05:09:09.420981 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:09.420983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:09.421042 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:09.421106 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:09.421513 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.422022 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.422784 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:09.423037 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:09.423112 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:09.425682 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:56875: BlockingRecv error: recv got EOF from 127.30.88.195:56875 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:10.076406 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:10.070754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:40977)
0811 05:09:10.071072 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:10.071088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:10.071202 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:10.071492 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:10.071495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:10.071700 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:10.071852 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.071864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.072728 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.072730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.073321 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.073388 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.073831 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.073833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.073900 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.074453 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:10.074464 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:10.076245 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":137,"threads_started":1}
W20260811 05:09:10.076540 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:10.070816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:40977 (local address 127.30.88.254:41699)
0811 05:09:10.071093 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:10.071096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:10.071150 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:09:10.071293 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:10.071295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:10.071345 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:10.071418 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:10.071983 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.072629 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.073502 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.073732 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.073812 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.076403 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:40977: BlockingRecv error: recv got EOF from 127.30.88.193:40977 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":92,"threads_started":1}
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:10.256273 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:10.250609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:50297)
0811 05:09:10.251030 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:10.251047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:10.251227 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:10.251521 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:10.251524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:10.251724 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:10.251807 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.251812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.252527 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.252532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.253065 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.253072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.253548 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.253551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.253662 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.254175 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:10.254187 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:10.256091 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":88,"threads_started":1}
W20260811 05:09:10.256291 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:10.250728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:50297 (local address 127.30.88.254:41699)
0811 05:09:10.250877 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:10.250881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:10.251110 (+ 229us) server_negotiation.cc:407] Connection header received
0811 05:09:10.251356 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:10.251359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:10.251411 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:10.251492 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:10.251917 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.252417 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.253208 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.253434 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.253542 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.256204 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:50297: BlockingRecv error: recv got EOF from 127.30.88.194:50297 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:10 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:10.434118 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:10.426610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:36573)
0811 05:09:10.426991 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:10.427007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:10.427118 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:10.427549 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:10.427551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:10.427821 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:10.427950 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.427957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.429024 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.429031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.430098 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:10.430112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.430832 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.430834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:10.430987 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.431555 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:10.431570 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:10.433992 (+ 2422us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":82,"threads_started":1}
W20260811 05:09:10.434274 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:10.426659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:36573 (local address 127.30.88.254:41699)
0811 05:09:10.426789 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:10.426793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:10.427076 (+ 283us) server_negotiation.cc:407] Connection header received
0811 05:09:10.427215 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:10.427219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:10.427268 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:10.427341 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:10.428083 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.428894 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.430304 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:10.430714 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:10.430838 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:10.434123 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:36573: BlockingRecv error: recv got EOF from 127.30.88.195:36573 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:11.082629 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:11.077272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:60351)
0811 05:09:11.077563 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:11.077577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:11.077690 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:11.077923 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:11.077925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:11.078112 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:11.078206 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.078212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.078984 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.078987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.079481 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.079488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.079865 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.079867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.079926 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.080401 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:11.080411 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:11.082494 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
W20260811 05:09:11.083006 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:11.077318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:60351 (local address 127.30.88.254:41699)
0811 05:09:11.077588 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:11.077592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:11.077642 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:09:11.077770 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:11.077772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:11.077814 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:11.077888 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:11.078310 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.078808 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.079574 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.079777 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.079848 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.082834 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:60351: BlockingRecv error: recv got EOF from 127.30.88.193:60351 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:11.263108 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:11.257220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:57497)
0811 05:09:11.257508 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:11.257521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:11.257658 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:11.257950 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:11.257953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:11.258170 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:11.258265 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.258271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.259168 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.259171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.260046 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.260056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.260623 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.260627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.260713 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.261182 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:11.261194 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:11.262981 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
W20260811 05:09:11.263226 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:11.257267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:57497 (local address 127.30.88.254:41699)
0811 05:09:11.257440 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:11.257446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:11.257599 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:09:11.257762 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:11.257765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:11.257819 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:11.257905 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:11.258388 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.259046 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.260203 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.260511 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.260617 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.263118 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:57497: BlockingRecv error: recv got EOF from 127.30.88.194:57497 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:11 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:11.440939 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:11.435247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:59309)
0811 05:09:11.435577 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:11.435595 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:11.435731 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:11.435992 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:11.435995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:11.436238 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:11.436359 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.436373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.437272 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.437277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.437864 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:11.437872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.438320 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.438323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:11.438406 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.438957 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:11.438975 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:11.440759 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
W20260811 05:09:11.440977 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:11.435364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59309 (local address 127.30.88.254:41699)
0811 05:09:11.435533 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:11.435537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:11.435670 (+ 133us) server_negotiation.cc:407] Connection header received
0811 05:09:11.435828 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:11.435831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:11.435885 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:11.435972 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:11.436516 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.437157 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.437991 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:11.438207 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:11.438316 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:11.440878 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:59309: BlockingRecv error: recv got EOF from 127.30.88.195:59309 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:12.089368 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:12.083642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:35813)
0811 05:09:12.084029 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:12.084043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:12.084186 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:12.084472 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:12.084474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:12.084668 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:12.084778 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.084782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.085759 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.085762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.086404 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.086414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.086903 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.086908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.086980 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.087453 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:12.087464 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:12.089232 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":97,"threads_started":1}
W20260811 05:09:12.089493 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:12.083751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:35813 (local address 127.30.88.254:41699)
0811 05:09:12.084024 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:12.084027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:12.084122 (+ 95us) server_negotiation.cc:407] Connection header received
0811 05:09:12.084291 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:12.084293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:12.084335 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:12.084413 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:12.084891 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.085623 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.086514 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.086751 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.086829 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.089360 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:35813: BlockingRecv error: recv got EOF from 127.30.88.193:35813 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:12.269757 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:12.264065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:60399)
0811 05:09:12.264359 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:12.264373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:12.264475 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:12.264709 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:12.264711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:12.264989 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:12.265088 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.265094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.265775 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.265778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.266335 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.266343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.266999 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.267003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.267089 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.267742 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:12.267756 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:12.269643 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
W20260811 05:09:12.269979 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:12.264117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:60399 (local address 127.30.88.254:41699)
0811 05:09:12.264264 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:12.264267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:12.264444 (+ 177us) server_negotiation.cc:407] Connection header received
0811 05:09:12.264556 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:12.264558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:12.264602 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:12.264674 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:12.265187 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.265664 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.266495 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.266836 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.266976 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.269823 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:60399: BlockingRecv error: recv got EOF from 127.30.88.194:60399 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:12 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:12.447121 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:12.441891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:43347)
0811 05:09:12.442227 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:12.442241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:12.442342 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:12.442581 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:12.442583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:12.442761 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:12.442906 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.442916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.443667 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.443669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.444225 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:12.444233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.444635 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.444637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:12.444711 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.445182 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:12.445192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:12.446986 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":137,"threads_started":1}
W20260811 05:09:12.447213 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:12.441974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:43347 (local address 127.30.88.254:41699)
0811 05:09:12.442137 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:12.442142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:12.442314 (+ 172us) server_negotiation.cc:407] Connection header received
0811 05:09:12.442423 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:12.442425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:12.442480 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:12.442545 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:12.443020 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.443573 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.444325 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:12.444541 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:12.444615 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:12.447112 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:43347: BlockingRecv error: recv got EOF from 127.30.88.195:43347 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:13.095428 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:13.090151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:35933)
0811 05:09:13.090442 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:13.090455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:13.090552 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:13.090797 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:13.090799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:13.091009 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:13.091109 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.091114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.091901 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.091904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.092494 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.092503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.092934 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.092937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.093001 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.093492 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:13.093503 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:13.095316 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
W20260811 05:09:13.095595 9540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:13.090293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:35933 (local address 127.30.88.254:41699)
0811 05:09:13.090516 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:13.090519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:13.090532 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:09:13.090638 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:13.090640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:13.090683 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:13.090760 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:13.091242 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.091797 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.092618 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.092830 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.092906 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.095434 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:35933: BlockingRecv error: recv got EOF from 127.30.88.193:35933 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":60,"threads_started":1}
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:13.276186 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:13.270468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:60571)
0811 05:09:13.270749 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:13.270763 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:13.270910 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:13.271190 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:13.271193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:13.271429 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:13.271529 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.271536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.272586 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.272589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.273206 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.273215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.273677 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.273679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.273740 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.274243 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:13.274254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:13.276076 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
W20260811 05:09:13.276348 9540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:13.270525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:60571 (local address 127.30.88.254:41699)
0811 05:09:13.270691 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:13.270695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:13.270838 (+ 143us) server_negotiation.cc:407] Connection header received
0811 05:09:13.271017 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:13.271020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:13.271069 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:13.271168 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:13.271664 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.272461 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.273307 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.273567 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.273645 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.276202 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:60571: BlockingRecv error: recv got EOF from 127.30.88.194:60571 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:13 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:13.453445 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:13.447940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:50637)
0811 05:09:13.448192 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:13.448205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:13.448318 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:13.448595 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:13.448598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:13.448779 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:13.448876 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.448881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.449590 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.449593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.450155 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:13.450162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.450610 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.450612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:13.450675 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.451213 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:13.451225 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:13.453333 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
W20260811 05:09:13.453601 9540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:13.448023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:50637 (local address 127.30.88.254:41699)
0811 05:09:13.448202 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:13.448205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:13.448285 (+ 80us) server_negotiation.cc:407] Connection header received
0811 05:09:13.448429 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:13.448431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:13.448489 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:13.448558 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:13.448973 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.449490 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.450282 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:13.450505 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:13.450593 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:13.453471 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:50637: BlockingRecv error: recv got EOF from 127.30.88.195:50637 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:14.106894 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:14.096198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:38791)
0811 05:09:14.096515 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:14.096530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:14.096653 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:14.096933 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:14.096935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:14.097134 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:14.097240 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.097246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.098013 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.098016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.098586 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.098596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.099030 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.099033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.099103 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.104518 (+ 5415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:14.104542 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:14.106673 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":72,"threads_started":1}
W20260811 05:09:14.106956 9544 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:14.096275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38791 (local address 127.30.88.254:41699)
0811 05:09:14.096558 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:14.096561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:14.096584 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:09:14.096752 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:14.096754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:14.096798 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:14.096889 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:14.097346 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.097873 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.098696 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.098932 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.099015 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.106798 (+ 7783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:38791: BlockingRecv error: recv got EOF from 127.30.88.193:38791 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:14.282565 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:14.276912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:39273)
0811 05:09:14.277233 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:14.277251 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:14.277398 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:14.277670 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:14.277674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:14.277897 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:14.278002 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.278008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.278813 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.278818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.279416 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.279424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.279888 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.279892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.279981 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.280558 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:14.280570 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:14.282430 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
W20260811 05:09:14.282699 9544 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:14.277053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:39273 (local address 127.30.88.254:41699)
0811 05:09:14.277232 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:14.277237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:14.277340 (+ 103us) server_negotiation.cc:407] Connection header received
0811 05:09:14.277502 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:14.277506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:14.277559 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:14.277645 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:14.278142 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.278698 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.279540 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.279769 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.279878 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.282560 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:39273: BlockingRecv error: recv got EOF from 127.30.88.194:39273 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:14 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:14.460194 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:14.454264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:50159)
0811 05:09:14.454591 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:14.454604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:14.454702 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:14.454935 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:14.454937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:14.455124 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:14.455218 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.455223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.456017 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.456019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.456579 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:14.456585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.457098 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.457101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:14.457167 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.457764 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:14.457781 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:14.460067 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":125,"threads_started":1}
W20260811 05:09:14.460274 9544 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:14.454308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:50159 (local address 127.30.88.254:41699)
0811 05:09:14.454464 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:14.454469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:14.454676 (+ 207us) server_negotiation.cc:407] Connection header received
0811 05:09:14.454776 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:14.454778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:14.454825 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:14.454907 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:14.455316 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.455921 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.456724 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:14.456998 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:14.457086 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:14.460184 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:50159: BlockingRecv error: recv got EOF from 127.30.88.195:50159 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:15.113168 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:15.107731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:48049)
0811 05:09:15.108005 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:15.108020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:15.108146 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:15.108382 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:15.108384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:15.108562 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:15.108678 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.108683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.109498 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.109501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.110144 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.110154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.110589 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.110591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.110655 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.111254 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:15.111279 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:15.113052 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
W20260811 05:09:15.113477 9548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:15.107760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:48049 (local address 127.30.88.254:41699)
0811 05:09:15.108006 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:15.108011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:15.108123 (+ 112us) server_negotiation.cc:407] Connection header received
0811 05:09:15.108226 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:15.108228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:15.108269 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:15.108341 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:15.108780 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.109300 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.110254 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.110488 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.110566 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.113217 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:48049: BlockingRecv error: recv got EOF from 127.30.88.193:48049 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:15.288290 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:15.283334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:50077)
0811 05:09:15.283591 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:15.283605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:15.283695 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:15.283906 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:15.283908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:15.284090 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:15.284184 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.284190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.284853 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.284855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.285388 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.285394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.285788 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.285791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.285852 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.286320 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:15.286332 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:15.288182 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
W20260811 05:09:15.288462 9548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:15.283418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:50077 (local address 127.30.88.254:41699)
0811 05:09:15.283552 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:15.283555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:15.283668 (+ 113us) server_negotiation.cc:407] Connection header received
0811 05:09:15.283758 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:15.283760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:15.283801 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:15.283875 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:15.284275 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.284768 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.285484 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.285693 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.285768 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.288316 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:50077: BlockingRecv error: recv got EOF from 127.30.88.194:50077 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:15 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:15.466804 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:15.461145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:54503)
0811 05:09:15.461473 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:15.461490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:15.461632 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:15.461945 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:15.461948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:15.462179 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:15.462273 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.462279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.463112 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.463118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.463790 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:15.463801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.464340 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.464349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:15.464453 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.464951 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:15.464962 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:15.466687 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":126,"threads_started":1}
W20260811 05:09:15.467129 9548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:15.461320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:54503 (local address 127.30.88.254:41699)
0811 05:09:15.461470 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:15.461475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:15.461573 (+ 98us) server_negotiation.cc:407] Connection header received
0811 05:09:15.461733 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:15.461737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:15.461794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:15.461911 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:15.462390 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.463005 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.463909 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:15.464222 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:15.464334 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:15.466842 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:54503: BlockingRecv error: recv got EOF from 127.30.88.195:54503 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:16.119833 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:16.113923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:53785)
0811 05:09:16.114502 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:16.114515 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:16.114610 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:16.114927 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:16.114930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:16.115127 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:16.115226 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.115231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.115988 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.115991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.116563 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.116571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.116997 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.116999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.117068 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.117599 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:16.117612 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:16.119644 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":387,"threads_started":1}
W20260811 05:09:16.119901 9552 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:16.113978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:53785 (local address 127.30.88.254:41699)
0811 05:09:16.114466 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:16.114478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:16.114582 (+ 104us) server_negotiation.cc:407] Connection header received
0811 05:09:16.114690 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:16.114692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:16.114735 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:16.114839 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:16.115329 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.115865 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.116674 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.116895 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.116976 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.119774 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:53785: BlockingRecv error: recv got EOF from 127.30.88.193:53785 (error 108)
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":333,"threads_started":1}
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:16.295034 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:16.289127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:44117)
0811 05:09:16.289445 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:16.289458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:16.289562 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:16.289800 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:16.289802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:16.289993 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:16.290108 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.290114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.290962 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.290965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.291543 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.291551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.292022 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.292026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.292104 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.292599 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:16.292610 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:16.294780 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":143,"threads_started":1}
W20260811 05:09:16.295047 9552 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:16.289228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:44117 (local address 127.30.88.254:41699)
0811 05:09:16.289378 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:16.289382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:16.289526 (+ 144us) server_negotiation.cc:407] Connection header received
0811 05:09:16.289640 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:16.289642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:16.289687 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:16.289760 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:16.290204 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.290804 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.291643 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.291899 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.291981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.294942 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:44117: BlockingRecv error: recv got EOF from 127.30.88.194:44117 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:16 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:16.472925 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:16.467585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:56727)
0811 05:09:16.467882 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:16.467895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:16.467996 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:16.468229 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:16.468231 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:16.468416 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:16.468528 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.468533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.469324 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.469327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.469890 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:16.469898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.470315 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.470318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:16.470385 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.470897 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:16.470918 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:16.472811 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
W20260811 05:09:16.473044 9552 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:16.467637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:56727 (local address 127.30.88.254:41699)
0811 05:09:16.467776 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:16.467781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:16.467967 (+ 186us) server_negotiation.cc:407] Connection header received
0811 05:09:16.468065 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:16.468067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:16.468115 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:16.468180 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:16.468652 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.469209 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.469987 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:16.470223 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:16.470296 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:16.472930 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:56727: BlockingRecv error: recv got EOF from 127.30.88.195:56727 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:17.126788 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:17.120944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:43043)
0811 05:09:17.121206 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:17.121219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:17.121319 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:17.121563 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:17.121564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:17.121759 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:17.121854 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.121859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.122744 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.122748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.123436 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.123465 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.124083 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.124086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.124168 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.124729 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:17.124741 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:17.126619 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
W20260811 05:09:17.126891 9556 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:17.120997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:43043 (local address 127.30.88.254:41699)
0811 05:09:17.121231 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:17.121236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:17.121295 (+ 59us) server_negotiation.cc:407] Connection header received
0811 05:09:17.121402 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:17.121404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:17.121449 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:17.121532 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:17.121982 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.122610 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.123618 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.123975 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.124071 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.126741 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:43043: BlockingRecv error: recv got EOF from 127.30.88.193:43043 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:17.300901 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:17.295749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:34925)
0811 05:09:17.296003 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:17.296016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:17.296111 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:17.296316 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:17.296318 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:17.296492 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:17.296586 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.296590 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.297328 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.297330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.297881 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.297888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.298289 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.298292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.298351 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.298839 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:17.298871 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:17.300718 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
W20260811 05:09:17.300938 9556 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:17.295794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:34925 (local address 127.30.88.254:41699)
0811 05:09:17.295951 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:17.295954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:17.296080 (+ 126us) server_negotiation.cc:407] Connection header received
0811 05:09:17.296187 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:17.296189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:17.296231 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:17.296305 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:17.296683 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.297225 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.297982 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.298192 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.298267 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.300836 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:34925: BlockingRecv error: recv got EOF from 127.30.88.194:34925 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:17 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:17.479143 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:17.473681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:55745)
0811 05:09:17.473977 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:17.473988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:17.474115 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:17.474395 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:17.474398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:17.474593 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:17.474684 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.474689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.475472 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.475476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.475981 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:17.475988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.476427 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.476430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:17.476516 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.477005 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:17.477015 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:17.479028 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":85,"threads_started":1}
W20260811 05:09:17.479271 9556 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:17.473731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:55745 (local address 127.30.88.254:41699)
0811 05:09:17.473899 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:17.473904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:17.474059 (+ 155us) server_negotiation.cc:407] Connection header received
0811 05:09:17.474219 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:17.474224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:17.474273 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:17.474350 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:17.474810 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.475368 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.476103 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:17.476315 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:17.476423 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:17.479140 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:55745: BlockingRecv error: recv got EOF from 127.30.88.195:55745 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:18.132877 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:18.127578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:60749)
0811 05:09:18.127861 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:18.127875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:18.127964 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:18.128238 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:18.128240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:18.128438 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:18.128558 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.128563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.129286 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.129288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.129835 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.129842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.130260 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.130262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.130334 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.130903 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:18.130915 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:18.132740 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
W20260811 05:09:18.133070 9562 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:18.127658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:60749 (local address 127.30.88.254:41699)
0811 05:09:18.127958 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:18.127961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:18.127974 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:09:18.128065 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:18.128067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:18.128109 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:18.128190 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:18.128657 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.129192 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.129936 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.130165 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.130241 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.132877 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:60749: BlockingRecv error: recv got EOF from 127.30.88.193:60749 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":103,"threads_started":1}
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:18.306638 9563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:18.301665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:60685)
0811 05:09:18.301938 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:18.301951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:18.302047 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:18.302256 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:18.302259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:18.302423 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:18.302525 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.302530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.303270 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.303273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.303806 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.303814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.304255 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.304258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.304316 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.304828 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:18.304840 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:18.306527 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":65,"threads_started":1}
W20260811 05:09:18.306903 9562 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:18.301918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:60685 (local address 127.30.88.254:41699)
0811 05:09:18.302043 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:18.302046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:18.302058 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:09:18.302123 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:18.302125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:18.302164 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:18.302216 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:18.302646 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.303173 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.303917 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.304164 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.304235 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.306655 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:60685: BlockingRecv error: recv got EOF from 127.30.88.194:60685 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:18 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:18.485580 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:18.479884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:59519)
0811 05:09:18.480287 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:18.480301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:18.480470 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:18.480754 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:18.480758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:18.480969 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:18.481069 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.481075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.481853 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.481857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.482394 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:18.482403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.482924 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.482928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:18.483023 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.483628 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:18.483641 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:18.485403 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":86,"threads_started":1}
W20260811 05:09:18.485615 9562 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:18.480003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59519 (local address 127.30.88.254:41699)
0811 05:09:18.480139 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:18.480144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:18.480420 (+ 276us) server_negotiation.cc:407] Connection header received
0811 05:09:18.480578 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:18.480581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:18.480640 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:18.480723 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:18.481190 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.481726 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.482517 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:18.482748 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:18.482916 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:18.485510 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:59519: BlockingRecv error: recv got EOF from 127.30.88.195:59519 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:19.143811 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:19.133734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:57007)
0811 05:09:19.134072 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:19.134086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:19.134203 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:19.134568 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:19.134571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:19.134821 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:19.134951 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.134958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.135680 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.135682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.136211 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.136218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.136608 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.136611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.136668 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.141839 (+ 5171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:19.141851 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:19.143688 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":93,"threads_started":1}
W20260811 05:09:19.143985 9566 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:19.133839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:57007 (local address 127.30.88.254:41699)
0811 05:09:19.134275 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:19.134302 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:19.134317 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:09:19.134357 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:19.134361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:19.134413 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:19.134508 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:19.135061 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.135582 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.136307 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.136514 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.136586 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.143834 (+ 7248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:57007: BlockingRecv error: recv got EOF from 127.30.88.193:57007 (error 108)
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":63,"threads_started":1}
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:19.312717 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:19.307512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:36683)
0811 05:09:19.307785 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:19.307799 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:19.307897 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:19.308138 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:19.308141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:19.308320 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:19.308433 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.308439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.309169 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.309171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.309737 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.309744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.310187 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.310189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.310253 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.310739 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:19.310751 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:19.312593 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":102,"threads_started":1}
W20260811 05:09:19.312841 9566 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:19.307603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:36683 (local address 127.30.88.254:41699)
0811 05:09:19.307746 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:19.307749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:19.307866 (+ 117us) server_negotiation.cc:407] Connection header received
0811 05:09:19.307978 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:19.307980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:19.308026 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:19.308102 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:19.308533 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.309077 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.309835 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.310057 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.310167 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.312725 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:36683: BlockingRecv error: recv got EOF from 127.30.88.194:36683 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:19 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.195@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:19.492134 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:19.486404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.195:56803)
0811 05:09:19.486708 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:19.486745 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:19.486928 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:19.487237 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:19.487239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:19.487519 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:19.487652 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.487659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.488383 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.488386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.488998 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:19.489006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.489407 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.489409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:19.489471 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.489951 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:19.489962 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:19.491984 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":121,"threads_started":1}
W20260811 05:09:19.492224 9566 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:19.486502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:56803 (local address 127.30.88.254:41699)
0811 05:09:19.486642 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:19.486649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:19.486845 (+ 196us) server_negotiation.cc:407] Connection header received
0811 05:09:19.487038 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:19.487041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:19.487098 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:19.487194 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:19.487756 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.488285 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.489106 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:19.489317 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:19.489387 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:19.492114 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.195:56803: BlockingRecv error: recv got EOF from 127.30.88.195:56803 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 05:09:20 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:20 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.193@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:20.150513 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:20.144765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.193:35133)
0811 05:09:20.145033 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:20.145046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:20.145126 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:20.145373 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:20.145375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:20.145570 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:20.145680 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.145685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.146558 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.146562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.147370 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.147383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.147878 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.147881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.147944 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.148433 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:20.148445 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:20.150386 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
W20260811 05:09:20.150707 9572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:20.144838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:35133 (local address 127.30.88.254:41699)
0811 05:09:20.145104 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:20.145108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:20.145121 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:09:20.145211 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:20.145213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:20.145254 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:20.145336 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:20.145806 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.146445 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.147530 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.147770 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.147852 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.150546 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.193:35133: BlockingRecv error: recv got EOF from 127.30.88.193:35133 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":100,"threads_started":1}
Aug 11 05:09:20 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:09:20 dist-test-slave-7c35 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.88.194@KRBTEST.COM for oryx/127.30.88.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:09:20.318548 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:20.313692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:41699 (local address 127.30.88.194:54109)
0811 05:09:20.313966 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:20.313979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:20.314065 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:20.314307 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:20.314309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:20.314490 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:20.314585 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.314590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.315299 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.315301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.315801 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.315809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.316203 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.316205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.316264 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.316716 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:20.316726 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:20.318426 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.88.254:41699: Server oryx/127.30.88.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
W20260811 05:09:20.318681 9572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:20.313843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:54109 (local address 127.30.88.254:41699)
0811 05:09:20.314018 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:20.314021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:20.314035 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:20.314153 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:20.314155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:20.314197 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:20.314266 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:20.314680 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.315215 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.315894 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.316091 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.316178 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.318556 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.88.194:54109: BlockingRecv error: recv got EOF from 127.30.88.194:54109 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 05:09:20.387802 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9036
I20260811 05:09:20.397712 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9215
I20260811 05:09:20.404752 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9363
I20260811 05:09:20.418404 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 8955
2026-08-11T05:09:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17966 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T05:09:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:20Z Disabled control of system clock
I20260811 05:09:20.450480 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37877
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:37877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:09:20.571650 9578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:20.571929 9578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:20.571960 9578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:20.575762 9578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:20.575809 9578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:20.575825 9578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:20.575841 9578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:20.575855 9578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:20.580314 9578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:37877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:37877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:20.581259 9578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:20.582482 9578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:20.590826 9585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:20.590821 9589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:20.590889 9578 server_base.cc:1034] running on GCE node
W20260811 05:09:20.590826 9584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:20.591598 9578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:20.592782 9578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:20.594010 9578 hybrid_clock.cc:648] HybridClock initialized: now 1786424960593990 us; error 38 us; skew 500 ppm
I20260811 05:09:20.596086 9578 webserver.cc:460] Webserver started at http://127.30.88.254:38995/ using document root <none> and password file <none>
I20260811 05:09:20.596578 9578 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:20.596653 9578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:20.596808 9578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:20.598697 9578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "ebebab24cdb340609e5d42312d1f1f0c"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "e12d0ed1a839ebb99442c638010f410c"
server_key_iv: "70c48933fa596411c9621eb18ad78dd4"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.599294 9578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "ebebab24cdb340609e5d42312d1f1f0c"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "e12d0ed1a839ebb99442c638010f410c"
server_key_iv: "70c48933fa596411c9621eb18ad78dd4"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.602676 9578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:09:20.605019 9598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:20.606182 9578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:20.606276 9578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "ebebab24cdb340609e5d42312d1f1f0c"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "e12d0ed1a839ebb99442c638010f410c"
server_key_iv: "70c48933fa596411c9621eb18ad78dd4"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.606356 9578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:20.637640 9578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:20.638278 9578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:20.638432 9578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:20.645118 9578 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:37877
I20260811 05:09:20.645133 9655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:37877 every 8 connection(s)
I20260811 05:09:20.646013 9578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:09:20.646348 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9578
I20260811 05:09:20.646502 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:09:20.646821 31075 external_mini_cluster.cc:1428] Setting key cb0724fb8213c193be68ec122b256b26
I20260811 05:09:20.649839 9656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:20.656625 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c: Bootstrap starting.
I20260811 05:09:20.656999 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:20.648287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.30.88.254:37877)
0811 05:09:20.648942 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:20.648962 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:20.649004 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:09:20.649835 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:20.649860 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:20.650369 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:20.650725 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:20.651182 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.652610 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.653423 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.654083 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.654246 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.655110 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:20.655116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:20.655131 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:20.655607 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:20.655679 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:20.655706 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:20.655804 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:20.655913 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 05:09:20.656157 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260811 05:09:20.658838 9656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:20.659585 9656 log.cc:824] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:20.661336 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c: No bootstrap required, opened a new log
I20260811 05:09:20.664911 9656 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } }
I20260811 05:09:20.665118 9656 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:20.665151 9656 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebebab24cdb340609e5d42312d1f1f0c, State: Initialized, Role: FOLLOWER
I20260811 05:09:20.665531 9656 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } }
I20260811 05:09:20.665645 9656 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:09:20.665689 9656 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:09:20.665755 9656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:20.666491 9656 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } }
I20260811 05:09:20.666749 9656 leader_election.cc:302] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebebab24cdb340609e5d42312d1f1f0c; no voters:
I20260811 05:09:20.667030 9656 leader_election.cc:288] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:09:20.667150 9661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:20.667369 9661 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [term 1 LEADER]: Becoming Leader. State: Replica: ebebab24cdb340609e5d42312d1f1f0c, State: Running, Role: LEADER
I20260811 05:09:20.667752 9661 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } }
I20260811 05:09:20.668169 9656 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:09:20.669387 9663 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebebab24cdb340609e5d42312d1f1f0c. Latest consensus state: current_term: 1 leader_uuid: "ebebab24cdb340609e5d42312d1f1f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } } }
I20260811 05:09:20.669389 9662 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebebab24cdb340609e5d42312d1f1f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebebab24cdb340609e5d42312d1f1f0c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 37877 } } }
I20260811 05:09:20.669605 9663 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:20.669633 9662 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:20.670472 9671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:09:20.673357 9671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:09:20.678519 9671 catalog_manager.cc:1347] Generated new cluster ID: 25449e3b549541a1be295f065a8fefaf
I20260811 05:09:20.678627 9671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:09:20.687716 9671 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:09:20.688788 9671 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:09:20.704016 9671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebebab24cdb340609e5d42312d1f1f0c: Generated new TSK 0
I20260811 05:09:20.704649 9671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:09:20.713027 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--builtin_ntp_servers=127.30.88.212:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 05:09:20.822839 9681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:20.823143 9681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:20.823177 9681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:20.827134 9681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:20.827181 9681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:20.827230 9681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:09:20.831800 9681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:20.832803 9681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:20.834151 9681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:20.843231 9687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:20.843225 9688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:20.843227 9690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:20.843482 9681 server_base.cc:1034] running on GCE node
I20260811 05:09:20.843890 9681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:20.844467 9681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:20.845670 9681 hybrid_clock.cc:648] HybridClock initialized: now 1786424960845657 us; error 24 us; skew 500 ppm
I20260811 05:09:20.847515 9681 webserver.cc:460] Webserver started at http://127.30.88.193:45313/ using document root <none> and password file <none>
I20260811 05:09:20.848074 9681 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:20.848165 9681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:20.848383 9681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:20.850371 9681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "e4ed98a087aa49e0b01dd5d1e1851624"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "72c6e0a52a7c7d786296953cf8122df3"
server_key_iv: "4105256766ce5e8bb83ed1daabafa614"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.850983 9681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "e4ed98a087aa49e0b01dd5d1e1851624"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "72c6e0a52a7c7d786296953cf8122df3"
server_key_iv: "4105256766ce5e8bb83ed1daabafa614"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.854300 9681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:09:20.856769 9696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:20.857930 9681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:20.858062 9681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "e4ed98a087aa49e0b01dd5d1e1851624"
format_stamp: "Formatted at 2026-08-11 05:09:20 on dist-test-slave-7c35"
server_key: "72c6e0a52a7c7d786296953cf8122df3"
server_key_iv: "4105256766ce5e8bb83ed1daabafa614"
server_key_version: "encryptionkey@0"
I20260811 05:09:20.858174 9681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:20.880020 9681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:20.880856 9681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:20.881098 9681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:20.881693 9681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:20.882602 9681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:20.882674 9681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:20.882737 9681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:20.882788 9681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:20.891644 9681 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:45713
I20260811 05:09:20.891673 9822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:45713 every 8 connection(s)
I20260811 05:09:20.892467 9681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:09:20.900080 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9681
I20260811 05:09:20.900192 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:09:20.900590 31075 external_mini_cluster.cc:1428] Setting key 58ecca8f0056575248bcbf16d23807d9
I20260811 05:09:20.901307 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:20.894104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:51043 (local address 127.30.88.254:37877)
0811 05:09:20.894233 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:20.894237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:20.895171 (+ 934us) server_negotiation.cc:407] Connection header received
0811 05:09:20.896518 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:20.896522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:20.896584 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:20.896675 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:20.897472 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.898021 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.899169 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.899416 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.899525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.900735 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:20.900739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:20.900741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:20.900753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:20.900771 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:20.900779 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:20.900841 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:20.901043 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 05:09:20.901125 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:09:20.902150 9826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:20.894394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37877 (local address 127.30.88.193:51043)
0811 05:09:20.895034 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:20.895072 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:20.896311 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:20.896832 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:20.896838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:20.896914 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:20.897327 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.897339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.898151 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.898157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.899030 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:20.899038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:20.899562 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:20.899566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:20.899722 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:20.900412 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:20.900415 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:20.900471 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:20.900558 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:20.900870 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:20.900888 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:20.900894 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:09:20.901028 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 05:09:20.901316 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260811 05:09:20.902833 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--builtin_ntp_servers=127.30.88.212:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 05:09:20.903342 9823 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37877
I20260811 05:09:20.903596 9823 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:20.904088 9823 heartbeater.cc:511] Master 127.30.88.254:37877 requested a full tablet report, sending...
I20260811 05:09:20.905645 9618 ts_manager.cc:194] Registered new tserver with Master: e4ed98a087aa49e0b01dd5d1e1851624 (127.30.88.193:45713)
I20260811 05:09:20.906759 9618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.193:51043
W20260811 05:09:21.010177 9827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:21.010392 9827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:21.010468 9827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:21.014053 9827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:21.014094 9827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:21.014161 9827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:09:21.018625 9827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:21.019551 9827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:21.020668 9827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:21.029109 9838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.029129 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.029107 9836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.029459 9827 server_base.cc:1034] running on GCE node
I20260811 05:09:21.029800 9827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:21.030405 9827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:21.031561 9827 hybrid_clock.cc:648] HybridClock initialized: now 1786424961031535 us; error 37 us; skew 500 ppm
I20260811 05:09:21.033310 9827 webserver.cc:460] Webserver started at http://127.30.88.194:35417/ using document root <none> and password file <none>
I20260811 05:09:21.033843 9827 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:21.033932 9827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:21.034085 9827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:21.035989 9827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "0e78e2615eab4f05a2e3b11ca618913d"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "158c06af393f5e53730a95606ea77e06"
server_key_iv: "6ed3c66d5450d2d692e7916903654c1d"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.036479 9827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "0e78e2615eab4f05a2e3b11ca618913d"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "158c06af393f5e53730a95606ea77e06"
server_key_iv: "6ed3c66d5450d2d692e7916903654c1d"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.040011 9827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:09:21.042281 9844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.043520 9827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:09:21.043612 9827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "0e78e2615eab4f05a2e3b11ca618913d"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "158c06af393f5e53730a95606ea77e06"
server_key_iv: "6ed3c66d5450d2d692e7916903654c1d"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.043722 9827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:21.058053 9827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:21.058599 9827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:21.058736 9827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:21.059273 9827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:21.060159 9827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:21.060204 9827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.060240 9827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:21.060261 9827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.069451 9974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:40867 every 8 connection(s)
I20260811 05:09:21.069453 9827 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:40867
I20260811 05:09:21.070426 9827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:09:21.079142 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.072136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:56919 (local address 127.30.88.254:37877)
0811 05:09:21.072303 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.072308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.073216 (+ 908us) server_negotiation.cc:407] Connection header received
0811 05:09:21.074419 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.074421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.074466 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.074534 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.075417 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.075908 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.077074 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.077395 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.077498 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:21.078613 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:21.078617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:21.078619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:21.078632 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:21.078656 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:21.078664 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:21.078738 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:21.078953 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 05:09:21.079011 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:09:21.079795 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9827
I20260811 05:09:21.079911 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:09:21.080021 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.072502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37877 (local address 127.30.88.194:56919)
0811 05:09:21.073081 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:21.073116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:21.074246 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:21.074695 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:21.074701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:21.074777 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:21.075286 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:21.075298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.076021 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.076024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:21.076886 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:21.076898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.077532 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.077553 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:21.077753 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:21.078336 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:21.078338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:21.078393 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:21.078451 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:21.078741 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:21.078763 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:21.078770 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:09:21.078950 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 05:09:21.079182 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260811 05:09:21.080225 31075 external_mini_cluster.cc:1428] Setting key 3fa62c85131574795920bf4a448d542c
I20260811 05:09:21.081254 9975 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37877
I20260811 05:09:21.081493 9975 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:21.081988 9975 heartbeater.cc:511] Master 127.30.88.254:37877 requested a full tablet report, sending...
I20260811 05:09:21.082154 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--builtin_ntp_servers=127.30.88.212:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 05:09:21.083042 9618 ts_manager.cc:194] Registered new tserver with Master: 0e78e2615eab4f05a2e3b11ca618913d (127.30.88.194:40867)
I20260811 05:09:21.083786 9618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.194:56919
W20260811 05:09:21.192338 9979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:21.192577 9979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:21.192611 9979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:21.196645 9979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:21.196693 9979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:21.196774 9979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:21.201349 9979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:37877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:21.202267 9979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:21.203614 9979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:21.213148 9984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.213141 9985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.213158 9987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.213457 9979 server_base.cc:1034] running on GCE node
I20260811 05:09:21.213841 9979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:21.214429 9979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:21.215590 9979 hybrid_clock.cc:648] HybridClock initialized: now 1786424961215563 us; error 37 us; skew 500 ppm
I20260811 05:09:21.217300 9979 webserver.cc:460] Webserver started at http://127.30.88.195:40487/ using document root <none> and password file <none>
I20260811 05:09:21.217900 9979 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:21.217969 9979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:21.218189 9979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:21.220247 9979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "677db22377894d9a9fd0187c145bbc82"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "c76463ae86678c141e62ef15d4552517"
server_key_iv: "effa85e6025256c4b3bbfb67fb72a615"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.220855 9979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "677db22377894d9a9fd0187c145bbc82"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "c76463ae86678c141e62ef15d4552517"
server_key_iv: "effa85e6025256c4b3bbfb67fb72a615"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.224436 9979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 05:09:21.226749 9993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.227953 9979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:09:21.228075 9979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "677db22377894d9a9fd0187c145bbc82"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "c76463ae86678c141e62ef15d4552517"
server_key_iv: "effa85e6025256c4b3bbfb67fb72a615"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.228176 9979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:21.247360 9979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:21.248041 9979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:21.248219 9979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:21.248757 9979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:21.249646 9979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:21.249692 9979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.249758 9979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:21.249795 9979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.258476 9979 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:40973
I20260811 05:09:21.258530 10119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:40973 every 8 connection(s)
I20260811 05:09:21.259389 9979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:21.268224 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.261123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:41193 (local address 127.30.88.254:37877)
0811 05:09:21.261303 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.261307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.262270 (+ 963us) server_negotiation.cc:407] Connection header received
0811 05:09:21.263721 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.263724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.263771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.263845 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.264684 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.265210 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.266294 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.266502 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.266580 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:21.267800 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:21.267803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:21.267805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:21.267817 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:21.267837 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:21.267844 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:21.267910 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:21.268063 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:09:21.268112 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:09:21.269155 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.261528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:37877 (local address 127.30.88.195:41193)
0811 05:09:21.262127 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:21.262163 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:21.263535 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:21.264004 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:21.264010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:21.264104 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:21.264549 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:21.264560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.265324 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.265326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:21.266187 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:21.266196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:21.266666 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:21.266670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:21.266817 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:21.267543 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:21.267545 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:21.267583 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:21.267651 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:21.267932 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:21.267939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:21.267943 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:09:21.268046 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:09:21.268254 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260811 05:09:21.269670 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 9979
I20260811 05:09:21.269795 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:09:21.270192 31075 external_mini_cluster.cc:1428] Setting key ed4e4984ac4da63e3448c53ffe7f0f3d
I20260811 05:09:21.270430 10120 heartbeater.cc:347] Connected to a master server at 127.30.88.254:37877
I20260811 05:09:21.270690 10120 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:21.271306 10120 heartbeater.cc:511] Master 127.30.88.254:37877 requested a full tablet report, sending...
I20260811 05:09:21.272390 9616 ts_manager.cc:194] Registered new tserver with Master: 677db22377894d9a9fd0187c145bbc82 (127.30.88.195:40973)
I20260811 05:09:21.272995 9616 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.195:41193
I20260811 05:09:21.284080 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:09:21.286654 9659 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:21.285936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.30.88.254:37877)
0811 05:09:21.286047 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.286051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.286204 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:09:21.286283 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.286285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.286320 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.286383 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.286553 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44442: BlockingRecv error: recv got EOF from 127.0.0.1:44442 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:09:21.286846 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9681
I20260811 05:09:21.303278 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9827
I20260811 05:09:21.315466 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9979
I20260811 05:09:21.324903 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 9578
2026-08-11T05:09:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (905 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T05:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:21Z Disabled control of system clock
I20260811 05:09:21.356504 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38363
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:34397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:09:21.465131 10133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:21.465359 10133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:21.465402 10133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:21.469209 10133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:21.469259 10133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:21.469273 10133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:21.469288 10133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:21.469302 10133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:21.473757 10133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:38363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:21.474653 10133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:21.475948 10133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:21.484133 10141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.484208 10138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.484419 10133 server_base.cc:1034] running on GCE node
W20260811 05:09:21.484162 10139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.485066 10133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:21.485980 10133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:21.487142 10133 hybrid_clock.cc:648] HybridClock initialized: now 1786424961487113 us; error 39 us; skew 500 ppm
I20260811 05:09:21.488951 10133 webserver.cc:460] Webserver started at http://127.30.88.254:39773/ using document root <none> and password file <none>
I20260811 05:09:21.489521 10133 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:21.489579 10133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:21.489810 10133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:21.491839 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "b7a302a1e47445eeb5dde5243c542906"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "b47c1aa16b395e550b677e061c98aa76"
server_key_iv: "a26365f73bc25fe27255658e140e4ae4"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.492383 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "b7a302a1e47445eeb5dde5243c542906"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "b47c1aa16b395e550b677e061c98aa76"
server_key_iv: "a26365f73bc25fe27255658e140e4ae4"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.495747 10133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:09:21.497972 10147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.499151 10133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:09:21.499274 10133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "b7a302a1e47445eeb5dde5243c542906"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "b47c1aa16b395e550b677e061c98aa76"
server_key_iv: "a26365f73bc25fe27255658e140e4ae4"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.499372 10133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:21.503584 10133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:21.504141 10133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:21.504317 10133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:21.510828 10133 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:38363
I20260811 05:09:21.510836 10205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:38363 every 8 connection(s)
I20260811 05:09:21.511787 10133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:09:21.514693 10206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:21.520299 10206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906: Bootstrap starting.
I20260811 05:09:21.521927 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10133
I20260811 05:09:21.522084 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:09:21.522351 31075 external_mini_cluster.cc:1428] Setting key 9e56308b4113747f214d542c36b2805c
I20260811 05:09:21.522652 10206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:21.523516 10206 log.cc:824] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:21.525257 10206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906: No bootstrap required, opened a new log
I20260811 05:09:21.527343 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.523480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.30.88.254:38363)
0811 05:09:21.524093 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.524108 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.524140 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:09:21.524711 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.524726 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.524991 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.525244 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.526071 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:21.526075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:21.526087 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:21.526095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:21.526145 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:21.526161 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:21.526246 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:21.526385 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:09:21.526634 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":145,"threads_started":1}
I20260811 05:09:21.529317 10206 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } }
I20260811 05:09:21.529584 10206 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:21.529639 10206 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a302a1e47445eeb5dde5243c542906, State: Initialized, Role: FOLLOWER
I20260811 05:09:21.529978 10206 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } }
I20260811 05:09:21.530108 10206 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:09:21.530171 10206 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:09:21.530237 10206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:21.531036 10206 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } }
I20260811 05:09:21.531304 10206 leader_election.cc:302] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7a302a1e47445eeb5dde5243c542906; no voters:
I20260811 05:09:21.531535 10206 leader_election.cc:288] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:09:21.531647 10212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:21.531857 10212 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [term 1 LEADER]: Becoming Leader. State: Replica: b7a302a1e47445eeb5dde5243c542906, State: Running, Role: LEADER
I20260811 05:09:21.532193 10212 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } }
I20260811 05:09:21.532681 10206 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:09:21.533784 10213 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7a302a1e47445eeb5dde5243c542906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } } }
I20260811 05:09:21.533906 10213 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:21.534222 10214 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7a302a1e47445eeb5dde5243c542906. Latest consensus state: current_term: 1 leader_uuid: "b7a302a1e47445eeb5dde5243c542906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a302a1e47445eeb5dde5243c542906" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 38363 } } }
I20260811 05:09:21.534379 10214 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906 [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:21.534299 10222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:09:21.537660 10222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:09:21.542722 10222 catalog_manager.cc:1347] Generated new cluster ID: 3f9f59a59e5c40dd89c4c5faed62f158
I20260811 05:09:21.542826 10222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:09:21.556981 10222 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:09:21.557260 10222 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:09:21.560660 10222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7a302a1e47445eeb5dde5243c542906: Generated new TSK 0
I20260811 05:09:21.561182 10222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:09:21.564286 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--builtin_ntp_servers=127.30.88.212:34397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:09:21.675980 10233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:21.676227 10233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:21.676292 10233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:21.680186 10233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:21.680246 10233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:21.680325 10233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:09:21.684605 10233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:21.685515 10233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:21.686744 10233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:21.695626 10241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.695648 10238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.695647 10239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.695947 10233 server_base.cc:1034] running on GCE node
I20260811 05:09:21.696341 10233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:21.696858 10233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:21.698019 10233 hybrid_clock.cc:648] HybridClock initialized: now 1786424961698002 us; error 31 us; skew 500 ppm
I20260811 05:09:21.699793 10233 webserver.cc:460] Webserver started at http://127.30.88.193:38267/ using document root <none> and password file <none>
I20260811 05:09:21.700361 10233 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:21.700415 10233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:21.700614 10233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:21.702440 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "df0159eca27d429b805f42b0c9ba00c9"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "3f00bad5fac509219aa47f614647a13f"
server_key_iv: "a0e185347eaa0007f634503d1b37fcd2"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.703038 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "df0159eca27d429b805f42b0c9ba00c9"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "3f00bad5fac509219aa47f614647a13f"
server_key_iv: "a0e185347eaa0007f634503d1b37fcd2"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.706285 10233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:09:21.708541 10248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.709692 10233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:09:21.709817 10233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "df0159eca27d429b805f42b0c9ba00c9"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "3f00bad5fac509219aa47f614647a13f"
server_key_iv: "a0e185347eaa0007f634503d1b37fcd2"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.709921 10233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:21.714187 10233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:21.714744 10233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:21.714962 10233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:21.715467 10233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:21.716274 10233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:21.716342 10233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.716408 10233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:21.716454 10233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.724915 10233 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:40395
I20260811 05:09:21.724967 10371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:40395 every 8 connection(s)
I20260811 05:09:21.725718 10233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:09:21.730772 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10233
I20260811 05:09:21.730932 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:09:21.731292 31075 external_mini_cluster.cc:1428] Setting key 152a90ffd0ef230bb08e554b6c6d8b15
I20260811 05:09:21.731508 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.727398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:37603 (local address 127.30.88.254:38363)
0811 05:09:21.727525 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.727529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.728483 (+ 954us) server_negotiation.cc:407] Connection header received
0811 05:09:21.729736 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.729739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.729783 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.729890 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.731038 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:21.731042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:21.731050 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:21.731063 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:21.731080 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:21.731088 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:21.731167 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:21.731329 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 05:09:21.731390 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:09:21.732266 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.727752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38363 (local address 127.30.88.193:37603)
0811 05:09:21.728341 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:21.728384 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:21.729525 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:21.730038 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:21.730044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:21.730131 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:21.730643 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:21.730647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:21.730697 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:21.730784 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:21.731179 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:21.731189 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:21.731195 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:09:21.731376 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 05:09:21.731589 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260811 05:09:21.733237 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--builtin_ntp_servers=127.30.88.212:34397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 05:09:21.733489 10372 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38363
I20260811 05:09:21.733705 10372 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:21.734154 10372 heartbeater.cc:511] Master 127.30.88.254:38363 requested a full tablet report, sending...
I20260811 05:09:21.735625 10165 ts_manager.cc:194] Registered new tserver with Master: df0159eca27d429b805f42b0c9ba00c9 (127.30.88.193:40395)
W20260811 05:09:21.840003 10375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:21.840240 10375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:21.840276 10375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:21.844233 10375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:21.844278 10375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:21.844353 10375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:09:21.848919 10375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:21.849833 10375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:21.851220 10375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:21.860538 10381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.860560 10384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:21.860571 10382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:21.861192 10375 server_base.cc:1034] running on GCE node
I20260811 05:09:21.861569 10375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:21.862200 10375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:21.863361 10375 hybrid_clock.cc:648] HybridClock initialized: now 1786424961863348 us; error 23 us; skew 500 ppm
I20260811 05:09:21.864982 10375 webserver.cc:460] Webserver started at http://127.30.88.194:45845/ using document root <none> and password file <none>
I20260811 05:09:21.865523 10375 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:21.865592 10375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:21.865852 10375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:21.867763 10375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "f521601e214c4cff9e587951d059e08a"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "ec1ec0e1ba67a794c3f0699401658458"
server_key_iv: "8a714b82ab8f4516c320ead0bf551e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.868302 10375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "f521601e214c4cff9e587951d059e08a"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "ec1ec0e1ba67a794c3f0699401658458"
server_key_iv: "8a714b82ab8f4516c320ead0bf551e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.871814 10375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:09:21.874086 10390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.875284 10375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:09:21.875433 10375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "f521601e214c4cff9e587951d059e08a"
format_stamp: "Formatted at 2026-08-11 05:09:21 on dist-test-slave-7c35"
server_key: "ec1ec0e1ba67a794c3f0699401658458"
server_key_iv: "8a714b82ab8f4516c320ead0bf551e82"
server_key_version: "encryptionkey@0"
I20260811 05:09:21.875540 10375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:21.879874 10375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:21.880410 10375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:21.880574 10375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:21.881093 10375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:21.881850 10375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:21.881906 10375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.881958 10375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:21.881980 10375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:21.890234 10375 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:32895
I20260811 05:09:21.890278 10522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:32895 every 8 connection(s)
I20260811 05:09:21.891076 10375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:09:21.896507 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.892759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:43333 (local address 127.30.88.254:38363)
0811 05:09:21.892906 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:21.892911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:21.893818 (+ 907us) server_negotiation.cc:407] Connection header received
0811 05:09:21.895020 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:21.895023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:21.895061 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:21.895135 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:21.896114 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:21.896116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:21.896118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:21.896130 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:21.896151 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:21.896160 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:21.896233 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:21.896369 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 05:09:21.896406 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:09:21.897295 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:21.893115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38363 (local address 127.30.88.194:43333)
0811 05:09:21.893683 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:21.893718 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:21.894791 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:21.895280 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:21.895286 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:21.895361 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:21.895858 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:21.895860 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:21.895892 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:21.895971 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:21.896239 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:21.896245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:21.896249 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:09:21.896385 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:09:21.896578 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":95,"threads_started":1}
I20260811 05:09:21.898403 10523 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38363
I20260811 05:09:21.898628 10523 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:21.899037 10523 heartbeater.cc:511] Master 127.30.88.254:38363 requested a full tablet report, sending...
I20260811 05:09:21.899983 10165 ts_manager.cc:194] Registered new tserver with Master: f521601e214c4cff9e587951d059e08a (127.30.88.194:32895)
I20260811 05:09:21.900528 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10375
I20260811 05:09:21.900662 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:09:21.900943 31075 external_mini_cluster.cc:1428] Setting key c634eacb904d8dbee9da43be2b4fae72
I20260811 05:09:21.902734 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--builtin_ntp_servers=127.30.88.212:34397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:09:22.011637 10526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:22.011850 10526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:22.011890 10526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:22.015651 10526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:22.015703 10526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:22.015771 10526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:22.020010 10526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:34397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:22.020929 10526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:22.022130 10526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:22.031137 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:22.031123 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:22.031140 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:22.031517 10526 server_base.cc:1034] running on GCE node
I20260811 05:09:22.031849 10526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:22.032405 10526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:22.033572 10526 hybrid_clock.cc:648] HybridClock initialized: now 1786424962033566 us; error 49 us; skew 500 ppm
I20260811 05:09:22.035295 10526 webserver.cc:460] Webserver started at http://127.30.88.195:38161/ using document root <none> and password file <none>
I20260811 05:09:22.035802 10526 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:22.035864 10526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:22.036134 10526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:22.037971 10526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "21efb460f3ea497883bd3e2a265492fa"
format_stamp: "Formatted at 2026-08-11 05:09:22 on dist-test-slave-7c35"
server_key: "1bcf8b4c0c66c44254c2df29784175f6"
server_key_iv: "e4a3014a0e794b3f06a1787b316dd5d2"
server_key_version: "encryptionkey@0"
I20260811 05:09:22.038522 10526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "21efb460f3ea497883bd3e2a265492fa"
format_stamp: "Formatted at 2026-08-11 05:09:22 on dist-test-slave-7c35"
server_key: "1bcf8b4c0c66c44254c2df29784175f6"
server_key_iv: "e4a3014a0e794b3f06a1787b316dd5d2"
server_key_version: "encryptionkey@0"
I20260811 05:09:22.041954 10526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:09:22.044292 10542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:22.045496 10526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:09:22.045635 10526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "21efb460f3ea497883bd3e2a265492fa"
format_stamp: "Formatted at 2026-08-11 05:09:22 on dist-test-slave-7c35"
server_key: "1bcf8b4c0c66c44254c2df29784175f6"
server_key_iv: "e4a3014a0e794b3f06a1787b316dd5d2"
server_key_version: "encryptionkey@0"
I20260811 05:09:22.045745 10526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:22.050374 10526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:22.051023 10526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:22.051211 10526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:22.051780 10526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:22.052677 10526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:22.052724 10526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:22.052788 10526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:22.052831 10526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:22.062265 10526 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:43589
I20260811 05:09:22.062280 10673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:43589 every 8 connection(s)
I20260811 05:09:22.063127 10526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:22.068743 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:22.064721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:36447 (local address 127.30.88.254:38363)
0811 05:09:22.064842 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:22.064845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:22.065759 (+ 914us) server_negotiation.cc:407] Connection header received
0811 05:09:22.066902 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:22.066906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:22.066954 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:22.067039 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:22.068287 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:22.068290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:22.068292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:09:22.068305 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:22.068328 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:09:22.068340 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:22.068430 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:22.068579 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 05:09:22.068632 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:09:22.069258 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10526
I20260811 05:09:22.069363 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:09:22.069660 31075 external_mini_cluster.cc:1428] Setting key 31e5a166264cee687ee8f503526b5fdc
I20260811 05:09:22.069702 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:22.065023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:38363 (local address 127.30.88.195:36447)
0811 05:09:22.065624 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:22.065662 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:22.066683 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:22.067259 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:22.067265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:22.067383 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:22.068019 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:09:22.068020 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:22.068057 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:09:22.068121 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:22.068433 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:22.068441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:22.068445 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:09:22.068577 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 05:09:22.068764 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260811 05:09:22.071002 10674 heartbeater.cc:347] Connected to a master server at 127.30.88.254:38363
I20260811 05:09:22.071228 10674 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:22.071630 10674 heartbeater.cc:511] Master 127.30.88.254:38363 requested a full tablet report, sending...
I20260811 05:09:22.072589 10165 ts_manager.cc:194] Registered new tserver with Master: 21efb460f3ea497883bd3e2a265492fa (127.30.88.195:43589)
I20260811 05:09:22.083665 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:09:22.086377 10210 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:09:22.085643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.30.88.254:38363)
0811 05:09:22.085813 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:22.085818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:22.085955 (+ 137us) server_negotiation.cc:407] Connection header received
0811 05:09:22.086038 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:22.086040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:22.086069 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:22.086132 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:22.086284 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55634: BlockingRecv error: recv got EOF from 127.0.0.1:55634 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 05:09:22.086596 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10233
I20260811 05:09:22.098769 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10375
I20260811 05:09:22.113085 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10526
I20260811 05:09:22.119843 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10133
2026-08-11T05:09:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (796 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:09:22 dist-test-slave-7c35 krb5kdc[10685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:09:22 dist-test-slave-7c35 krb5kdc[10685](info): set up 2 sockets
Aug 11 05:09:22 dist-test-slave-7c35 krb5kdc[10685](info): commencing operation
krb5kdc: starting...
W20260811 05:09:23.189487 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:09:23.313994 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:40149
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:40149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:23.427714 10699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:23.427973 10699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:23.428015 10699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:23.432073 10699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:09:23.432134 10699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:23.432156 10699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:23.432176 10699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:09:23.432236 10699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:09:23.437157 10699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:40149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:40149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:23.438189 10699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:23.439254 10699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:23.447273 10707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:23.447300 10704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:23.447382 10699 server_base.cc:1034] running on GCE node
W20260811 05:09:23.447301 10705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:23.448118 10699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:23.449132 10699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:23.450323 10699 hybrid_clock.cc:648] HybridClock initialized: now 1786424963450290 us; error 44 us; skew 500 ppm
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:23.452943 10699 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:09:23.453749 10699 webserver.cc:460] Webserver started at http://127.30.88.254:34339/ using document root <none> and password file <none>
I20260811 05:09:23.454246 10699 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:23.454331 10699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:23.454571 10699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:23.456496 10699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "c013f5040ff6498ead867fe16e2a386f"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "a8211d0e225bb28549ae6417d235d7a1"
server_key_iv: "d2c0eb34436be8a51d1012b87816c041"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.457062 10699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "c013f5040ff6498ead867fe16e2a386f"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "a8211d0e225bb28549ae6417d235d7a1"
server_key_iv: "d2c0eb34436be8a51d1012b87816c041"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.460676 10699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:09:23.462981 10714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.464114 10699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:09:23.464255 10699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "c013f5040ff6498ead867fe16e2a386f"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "a8211d0e225bb28549ae6417d235d7a1"
server_key_iv: "d2c0eb34436be8a51d1012b87816c041"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.464383 10699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:23.482780 10699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:23.485724 10699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:23.485932 10699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:23.492987 10699 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:40149
I20260811 05:09:23.493248 10774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:40149 every 8 connection(s)
I20260811 05:09:23.493885 10699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:09:23.496464 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:23.500051 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10699
I20260811 05:09:23.500169 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:09:23.500453 31075 external_mini_cluster.cc:1428] Setting key 820b3724087198af63844e3df81ffd8b
I20260811 05:09:23.502059 10775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: Bootstrap starting.
I20260811 05:09:23.504463 10775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:23.505142 10775 log.cc:824] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:23.507057 10775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: No bootstrap required, opened a new log
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:23.511463 10775 raft_consensus.cc:348] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } }
I20260811 05:09:23.511710 10775 raft_consensus.cc:374] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:23.511770 10775 raft_consensus.cc:729] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c013f5040ff6498ead867fe16e2a386f, State: Initialized, Role: FOLLOWER
I20260811 05:09:23.512188 10775 consensus_queue.cc:260] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } }
I20260811 05:09:23.512343 10775 raft_consensus.cc:388] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:09:23.512408 10775 raft_consensus.cc:482] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:09:23.512504 10775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:23.513465 10775 raft_consensus.cc:504] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } }
I20260811 05:09:23.513774 10775 leader_election.cc:302] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c013f5040ff6498ead867fe16e2a386f; no voters:
I20260811 05:09:23.514053 10775 leader_election.cc:288] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:09:23.514194 10782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:23.514465 10782 raft_consensus.cc:686] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [term 1 LEADER]: Becoming Leader. State: Replica: c013f5040ff6498ead867fe16e2a386f, State: Running, Role: LEADER
I20260811 05:09:23.514551 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:23.501688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.30.88.254:40149)
0811 05:09:23.502060 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:23.502074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:23.502106 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:09:23.502805 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:23.502821 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:23.503184 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:23.503444 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:23.504295 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.505326 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.506303 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.506732 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.506939 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:23.509603 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:23.509612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:23.509629 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:23.509657 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:23.511606 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.512099 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.512104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.512109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.512238 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.512568 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.512572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.512573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.513033 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:23.513227 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:23.513406 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 05:09:23.513671 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
I20260811 05:09:23.514750 10782 consensus_queue.cc:237] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } }
I20260811 05:09:23.515169 10775 sys_catalog.cc:564] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:09:23.516304 10783 sys_catalog.cc:455] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c013f5040ff6498ead867fe16e2a386f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } } }
I20260811 05:09:23.516417 10783 sys_catalog.cc:458] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:23.516728 10784 sys_catalog.cc:455] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c013f5040ff6498ead867fe16e2a386f. Latest consensus state: current_term: 1 leader_uuid: "c013f5040ff6498ead867fe16e2a386f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c013f5040ff6498ead867fe16e2a386f" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40149 } } }
I20260811 05:09:23.516813 10784 sys_catalog.cc:458] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f [sys.catalog]: This master's current role is: LEADER
I20260811 05:09:23.517550 10791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:09:23.520005 10791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:09:23.524922 10791 catalog_manager.cc:1347] Generated new cluster ID: ebe38a07daa74dc0946ce1e0dc8edb31
I20260811 05:09:23.524978 10791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:09:23.534212 10791 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:09:23.535002 10791 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:09:23.543119 10791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: Generated new TSK 0
I20260811 05:09:23.543661 10791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:09:23.571516 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:23.684959 10805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:23.685181 10805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:23.685218 10805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:23.689267 10805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:23.689317 10805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:23.689409 10805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:09:23.694314 10805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:23.695230 10805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:23.696151 10805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:23.705293 10810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:23.705358 10805 server_base.cc:1034] running on GCE node
W20260811 05:09:23.705302 10811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:23.705302 10813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:23.705989 10805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:23.706527 10805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:23.707682 10805 hybrid_clock.cc:648] HybridClock initialized: now 1786424963707645 us; error 49 us; skew 500 ppm
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:23.710474 10805 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:09:23.711290 10805 webserver.cc:460] Webserver started at http://127.30.88.193:40877/ using document root <none> and password file <none>
I20260811 05:09:23.711763 10805 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:23.711817 10805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:23.711975 10805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:23.713924 10805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "b068a6a3469d406d885fee44e49d91bb"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "019518da175d858b3b7919183c866784"
server_key_iv: "f383020f030efbc550b95411821f987d"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.714424 10805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "b068a6a3469d406d885fee44e49d91bb"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "019518da175d858b3b7919183c866784"
server_key_iv: "f383020f030efbc550b95411821f987d"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.718197 10805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:09:23.720520 10820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.721627 10805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:09:23.721750 10805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "b068a6a3469d406d885fee44e49d91bb"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "019518da175d858b3b7919183c866784"
server_key_iv: "f383020f030efbc550b95411821f987d"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.721844 10805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:23.730461 10805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:23.733397 10805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:23.733572 10805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:23.734100 10805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:23.734802 10805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:23.734896 10805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.734958 10805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:23.734989 10805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.743177 10805 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:42787
I20260811 05:09:23.743229 10950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:42787 every 8 connection(s)
I20260811 05:09:23.743984 10805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:09:23.747520 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10805
I20260811 05:09:23.747646 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:09:23.747957 31075 external_mini_cluster.cc:1428] Setting key 2bbf32f03d77afa11153333216ac4dae
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:23.758404 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:23.745525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:53947 (local address 127.30.88.254:40149)
0811 05:09:23.745663 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:23.745667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:23.746411 (+ 744us) server_negotiation.cc:407] Connection header received
0811 05:09:23.747678 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:23.747681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:23.747736 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:23.747878 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:23.749343 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.750147 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.751327 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.751672 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.751756 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:23.754731 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:23.754735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:23.754739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:23.754770 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:23.756124 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.756749 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.756763 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.756765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.756830 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.757206 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.757209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.757210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.757438 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:23.757525 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:23.758139 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 05:09:23.758271 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:09:23.759500 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:23.745863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40149 (local address 127.30.88.193:53947)
0811 05:09:23.746255 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:23.746291 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:23.747489 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:23.748075 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:23.748084 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:23.748548 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:23.749164 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:23.749184 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.750276 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.750279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:23.751203 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:23.751213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.751821 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.751827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:23.752061 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:23.752609 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:23.752634 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:23.754550 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:23.756254 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:23.756261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:23.756278 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:23.756651 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:23.756916 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:23.756921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:23.756923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:23.757111 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:23.757555 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:23.757562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:23.757870 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 05:09:23.758105 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 05:09:23.758450 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260811 05:09:23.760625 10951 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40149
I20260811 05:09:23.760807 10951 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:23.761194 10951 heartbeater.cc:511] Master 127.30.88.254:40149 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:09:23.762743 10731 ts_manager.cc:194] Registered new tserver with Master: b068a6a3469d406d885fee44e49d91bb (127.30.88.193:42787)
I20260811 05:09:23.763996 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:53947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:09:23.776067 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:23.879292 10956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:23.879587 10956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:23.879671 10956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:23.883502 10956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:23.883584 10956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:23.883672 10956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:09:23.888477 10956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:23.889436 10956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:23.890419 10956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:23.899322 10964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:23.899288 10962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:23.899341 10961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:23.899668 10956 server_base.cc:1034] running on GCE node
I20260811 05:09:23.900025 10956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:23.900617 10956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:23.901808 10956 hybrid_clock.cc:648] HybridClock initialized: now 1786424963901798 us; error 27 us; skew 500 ppm
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:23.904254 10956 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:09:23.904981 10956 webserver.cc:460] Webserver started at http://127.30.88.194:33405/ using document root <none> and password file <none>
I20260811 05:09:23.905414 10956 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:23.905462 10956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:23.905613 10956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:23.907361 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "9973381e6f14435b8b75799616446f11"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "4d2576e37b2f8e01251c90da8b2192d7"
server_key_iv: "6644e0d14e789c41d1693adf7ac0b899"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.907819 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "9973381e6f14435b8b75799616446f11"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "4d2576e37b2f8e01251c90da8b2192d7"
server_key_iv: "6644e0d14e789c41d1693adf7ac0b899"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.911181 10956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:09:23.913219 10971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.914306 10956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:09:23.914400 10956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "9973381e6f14435b8b75799616446f11"
format_stamp: "Formatted at 2026-08-11 05:09:23 on dist-test-slave-7c35"
server_key: "4d2576e37b2f8e01251c90da8b2192d7"
server_key_iv: "6644e0d14e789c41d1693adf7ac0b899"
server_key_version: "encryptionkey@0"
I20260811 05:09:23.914476 10956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:23.932646 10956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:23.935601 10956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:23.935732 10956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:23.936247 10956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:23.936928 10956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:23.936970 10956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.937004 10956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:23.937018 10956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:23.946183 10956 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:38257
I20260811 05:09:23.946228 11098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:38257 every 8 connection(s)
I20260811 05:09:23.947021 10956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:09:23.951422 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 10956
I20260811 05:09:23.951541 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:09:23.951839 31075 external_mini_cluster.cc:1428] Setting key 670f5cc95105a42b0f36baf0a10bb8fd
Aug 11 05:09:23 dist-test-slave-7c35 krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424963, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:23.961239 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:23.948914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:46893 (local address 127.30.88.254:40149)
0811 05:09:23.949079 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:23.949083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:23.949826 (+ 743us) server_negotiation.cc:407] Connection header received
0811 05:09:23.951406 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:23.951410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:23.951473 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:23.951581 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:23.952860 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.953469 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.954526 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.954911 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.955025 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:23.957577 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:23.957581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:23.957585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:23.957614 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:23.959268 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.959774 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.959778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.959780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.959862 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:23.960196 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:23.960200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:23.960202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:23.960383 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:23.960490 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:23.960937 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 05:09:23.961056 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 05:09:23.961870 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:23.949219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40149 (local address 127.30.88.194:46893)
0811 05:09:23.949671 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:23.949712 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:23.951177 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:23.951772 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:23.951778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:23.952187 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:23.952726 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:23.952740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.953597 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.953600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:23.954396 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:23.954406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:23.955046 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:23.955049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:23.955249 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:23.955791 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:23.955814 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:23.957419 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:23.959401 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:23.959407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:23.959420 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:23.959669 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:23.959959 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:23.959963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:23.959965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:23.960100 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:23.960490 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:23.960497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:23.960716 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 05:09:23.960890 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 05:09:23.961175 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:09:23.962987 11099 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40149
I20260811 05:09:23.963176 11099 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:23.963688 11099 heartbeater.cc:511] Master 127.30.88.254:40149 requested a full tablet report, sending...
I20260811 05:09:23.964712 10731 ts_manager.cc:194] Registered new tserver with Master: 9973381e6f14435b8b75799616446f11 (127.30.88.194:38257)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:09:23.965397 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:46893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:09:23.980952 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:09:24.084281 11104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:09:24.084538 11104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:09:24.084609 11104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:09:24.088323 11104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:09:24.088405 11104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:09:24.088490 11104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:09:24.093453 11104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:09:24.094272 11104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:09:24.095157 11104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:09:24.104209 11109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:09:24.104218 11112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:24.104414 11104 server_base.cc:1034] running on GCE node
W20260811 05:09:24.104525 11110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:09:24.104935 11104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:09:24.105521 11104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:09:24.106707 11104 hybrid_clock.cc:648] HybridClock initialized: now 1786424964106677 us; error 41 us; skew 500 ppm
Aug 11 05:09:24 dist-test-slave-7c35 krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424964, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:09:24.109371 11104 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:09:24.110140 11104 webserver.cc:460] Webserver started at http://127.30.88.195:42135/ using document root <none> and password file <none>
I20260811 05:09:24.110591 11104 fs_manager.cc:359] Metadata directory not provided
I20260811 05:09:24.110647 11104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:09:24.110791 11104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:09:24.112645 11104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "7c8143a5b0414e64b2e909eb58586d83"
format_stamp: "Formatted at 2026-08-11 05:09:24 on dist-test-slave-7c35"
server_key: "87fce116209fdf4f3c261d885752c7ca"
server_key_iv: "d314a5ae2a2bfb61062633a698853d3b"
server_key_version: "encryptionkey@0"
I20260811 05:09:24.113128 11104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "7c8143a5b0414e64b2e909eb58586d83"
format_stamp: "Formatted at 2026-08-11 05:09:24 on dist-test-slave-7c35"
server_key: "87fce116209fdf4f3c261d885752c7ca"
server_key_iv: "d314a5ae2a2bfb61062633a698853d3b"
server_key_version: "encryptionkey@0"
I20260811 05:09:24.116870 11104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:09:24.119199 11119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:24.120271 11104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:09:24.120362 11104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "7c8143a5b0414e64b2e909eb58586d83"
format_stamp: "Formatted at 2026-08-11 05:09:24 on dist-test-slave-7c35"
server_key: "87fce116209fdf4f3c261d885752c7ca"
server_key_iv: "d314a5ae2a2bfb61062633a698853d3b"
server_key_version: "encryptionkey@0"
I20260811 05:09:24.120482 11104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:09:24.132069 11104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:09:24.134553 11104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:09:24.134727 11104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:09:24.135345 11104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:09:24.136087 11104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:09:24.136129 11104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:24.136190 11104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:09:24.136220 11104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:09:24.144905 11104 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:42643
I20260811 05:09:24.144949 11243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:42643 every 8 connection(s)
I20260811 05:09:24.145655 11104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:09:24.146586 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 11104
I20260811 05:09:24.146692 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:09:24.147009 31075 external_mini_cluster.cc:1428] Setting key add6cb3c0ab5f565160c37a27d78ede0
Aug 11 05:09:24 dist-test-slave-7c35 krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424964, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:09:24.165030 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.147358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:37691 (local address 127.30.88.254:40149)
0811 05:09:24.147501 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.147505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.148487 (+ 982us) server_negotiation.cc:407] Connection header received
0811 05:09:24.150057 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.150060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.150158 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.150244 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:24.151374 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.151946 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.153019 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.153246 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.153344 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.160712 (+ 7368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:24.160719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:24.160723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:24.160761 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:24.162541 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:24.163227 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:24.163234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:24.163237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:24.163348 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:24.163847 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:24.163851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:24.163853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:24.164058 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:24.164180 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.164722 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.164848 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:09:24.165796 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.147749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40149 (local address 127.30.88.195:37691)
0811 05:09:24.148301 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.148386 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.149842 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.150364 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.150370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.150696 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:09:24.151195 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.151209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.152071 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.152074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.152898 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.152908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.153370 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.153373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.153514 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.158753 (+ 5239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:09:24.158769 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:09:24.160511 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:09:24.162696 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:24.162703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:24.162717 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:24.163106 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:24.163461 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:09:24.163465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:09:24.163468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:09:24.163691 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:09:24.164193 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:09:24.164202 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:09:24.164448 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.164645 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.164974 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":152,"threads_started":1}
I20260811 05:09:24.166810 11244 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40149
I20260811 05:09:24.167088 11244 heartbeater.cc:464] Registering TS with master...
I20260811 05:09:24.167560 11244 heartbeater.cc:511] Master 127.30.88.254:40149 requested a full tablet report, sending...
I20260811 05:09:24.168737 10731 ts_manager.cc:194] Registered new tserver with Master: 7c8143a5b0414e64b2e909eb58586d83 (127.30.88.195:42643)
I20260811 05:09:24.169263 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:37691
I20260811 05:09:24.172828 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:09:24.181429 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.174712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.30.88.254:40149)
0811 05:09:24.174819 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.174822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.175025 (+ 203us) server_negotiation.cc:407] Connection header received
0811 05:09:24.175110 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.175112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.175149 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.175210 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:09:24.175842 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.176342 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.177109 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.177309 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.177382 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.178593 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:09:24.178596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:09:24.178598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:09:24.178619 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:09:24.180067 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:24.180433 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:24.180436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:24.180437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:24.180509 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:09:24.180735 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:09:24.180737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:09:24.180738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:09:24.180927 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:09:24.181026 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.181221 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.181308 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:09:24.184087 10731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:09:24.187079 10731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:09:24.201076 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.195382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.30.88.195:42643)
0811 05:09:24.195828 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.195838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.196298 (+ 460us) server_negotiation.cc:407] Connection header received
0811 05:09:24.196389 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.196395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.196610 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.196777 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:24.197060 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198182 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.199834 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.200391 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.200530 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.200677 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.200828 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.200921 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":173,"threads_started":1}
I20260811 05:09:24.201068 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.195146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:42643 (local address 127.0.0.1:55398)
0811 05:09:24.195943 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.196012 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.196149 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.196765 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.196771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.196793 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:24.196943 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.196950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198325 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.198329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.199715 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.199725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.200529 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.200533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.200640 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.200676 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.200792 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.200912 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":106,"threads_started":1}
I20260811 05:09:24.201891 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.196257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:42787 (local address 127.0.0.1:35556)
0811 05:09:24.197120 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.197134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.197214 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.197773 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.197775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.197801 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:24.197873 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.197879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198988 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.198990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.200489 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.200497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.201610 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.201613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.201677 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.201697 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.201746 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.201795 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":155,"thread_start_us":89,"threads_started":1}
I20260811 05:09:24.202026 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.196674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.30.88.193:42787)
0811 05:09:24.197237 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.197248 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.197274 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:09:24.197349 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.197354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.197538 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.197676 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:24.197955 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198803 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.200634 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.201473 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.201596 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.201728 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.201776 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.201859 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":164,"threads_started":1}
I20260811 05:09:24.203598 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.195924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:38257 (local address 127.0.0.1:49694)
0811 05:09:24.196178 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.196193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.196296 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.197518 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.197521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.197539 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:24.197640 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.197646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198959 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.198965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.200485 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.200497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.203201 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.203204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.203298 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.203320 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.203401 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.203465 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":161,"thread_start_us":103,"threads_started":1}
I20260811 05:09:24.203820 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.196116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.30.88.194:38257)
0811 05:09:24.197038 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.197056 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.197083 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:09:24.197168 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.197176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.197374 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.197538 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:24.197737 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.198775 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.200634 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.203064 (+ 2430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.203176 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.203306 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.203447 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.203541 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":155,"threads_started":1}
I20260811 05:09:24.204363 11171 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dbdb594594ad6a88519a159e20304 (DEFAULT_TABLE table=test-table [id=b28a4a4162964d43b7a8acec992bfb92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:24.204896 10873 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dbdb594594ad6a88519a159e20304 (DEFAULT_TABLE table=test-table [id=b28a4a4162964d43b7a8acec992bfb92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:24.205502 11171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dbdb594594ad6a88519a159e20304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:24.205902 10873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dbdb594594ad6a88519a159e20304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:24.206525 11024 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dbdb594594ad6a88519a159e20304 (DEFAULT_TABLE table=test-table [id=b28a4a4162964d43b7a8acec992bfb92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:09:24.207551 11024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dbdb594594ad6a88519a159e20304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:09:24.212266 11262 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Bootstrap starting.
I20260811 05:09:24.214155 11264 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Bootstrap starting.
I20260811 05:09:24.214521 11262 tablet_bootstrap.cc:654] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:24.215889 11262 log.cc:824] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:24.216647 11263 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Bootstrap starting.
I20260811 05:09:24.216991 11264 tablet_bootstrap.cc:654] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:24.217785 11264 log.cc:824] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:24.218120 11262 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: No bootstrap required, opened a new log
I20260811 05:09:24.218358 11262 ts_tablet_manager.cc:1397] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:09:24.219403 11263 tablet_bootstrap.cc:654] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Neither blocks nor log segments found. Creating new log.
I20260811 05:09:24.220027 11263 log.cc:824] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:09:24.221544 11264 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: No bootstrap required, opened a new log
I20260811 05:09:24.221738 11264 ts_tablet_manager.cc:1397] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 05:09:24.221572 11262 raft_consensus.cc:348] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.221828 11263 tablet_bootstrap.cc:492] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: No bootstrap required, opened a new log
I20260811 05:09:24.221815 11262 raft_consensus.cc:374] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:24.221879 11262 raft_consensus.cc:729] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8143a5b0414e64b2e909eb58586d83, State: Initialized, Role: FOLLOWER
I20260811 05:09:24.222059 11263 ts_tablet_manager.cc:1397] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:09:24.222328 11262 consensus_queue.cc:260] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.223033 11244 heartbeater.cc:503] Master 127.30.88.254:40149 was elected leader, sending a full tablet report...
I20260811 05:09:24.223332 11262 ts_tablet_manager.cc:1428] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:09:24.225651 11264 raft_consensus.cc:348] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.225689 11263 raft_consensus.cc:348] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.225869 11264 raft_consensus.cc:374] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:24.225915 11264 raft_consensus.cc:729] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9973381e6f14435b8b75799616446f11, State: Initialized, Role: FOLLOWER
I20260811 05:09:24.225977 11263 raft_consensus.cc:374] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:09:24.226056 11263 raft_consensus.cc:729] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b068a6a3469d406d885fee44e49d91bb, State: Initialized, Role: FOLLOWER
I20260811 05:09:24.226325 11264 consensus_queue.cc:260] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.226502 11263 consensus_queue.cc:260] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.227100 10951 heartbeater.cc:503] Master 127.30.88.254:40149 was elected leader, sending a full tablet report...
I20260811 05:09:24.227267 11264 ts_tablet_manager.cc:1428] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:09:24.227375 11263 ts_tablet_manager.cc:1428] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:09:24.227587 11099 heartbeater.cc:503] Master 127.30.88.254:40149 was elected leader, sending a full tablet report...
W20260811 05:09:24.244975 10952 tablet.cc:2369] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:09:24.313609 11268 raft_consensus.cc:482] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:09:24.313809 11268 raft_consensus.cc:504] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.315066 11268 leader_election.cc:288] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9973381e6f14435b8b75799616446f11 (127.30.88.194:38257), b068a6a3469d406d885fee44e49d91bb (127.30.88.193:42787)
I20260811 05:09:24.319881 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.315708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:38761 (local address 127.30.88.193:42787)
0811 05:09:24.315858 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.315862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.316130 (+ 268us) server_negotiation.cc:407] Connection header received
0811 05:09:24.316259 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.316262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.316310 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.316382 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:24.316624 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.317353 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.318776 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.319270 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.319365 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.319406 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.319657 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.319740 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:09:24.319885 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.315606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:42787 (local address 127.30.88.195:38761)
0811 05:09:24.316017 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.316027 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.316145 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.316407 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.316409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.316422 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:24.316512 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.316516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.317422 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.317425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.318634 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.318645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.319397 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.319401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.319515 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.319542 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.319619 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.319700 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":24,"thread_start_us":118,"threads_started":1}
I20260811 05:09:24.320024 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.315326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:38257 (local address 127.30.88.195:59671)
0811 05:09:24.315486 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.315617 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0811 05:09:24.315709 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:09:24.316046 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:09:24.316048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:09:24.316066 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:09:24.316178 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.316183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.317249 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.317252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.318634 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:09:24.318645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.319533 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.319536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:09:24.319672 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.319693 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:09:24.319769 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:09:24.319836 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":41}
I20260811 05:09:24.320042 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.315465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:59671 (local address 127.30.88.194:38257)
0811 05:09:24.315702 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.315706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.315720 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:09:24.315887 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.315891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.315949 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.316032 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:09:24.316294 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.317114 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.318971 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.319411 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.319527 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.319575 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.319805 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.319908 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260811 05:09:24.320691 11046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dbdb594594ad6a88519a159e20304" candidate_uuid: "7c8143a5b0414e64b2e909eb58586d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9973381e6f14435b8b75799616446f11" is_pre_election: true
I20260811 05:09:24.320716 10895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dbdb594594ad6a88519a159e20304" candidate_uuid: "7c8143a5b0414e64b2e909eb58586d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b068a6a3469d406d885fee44e49d91bb" is_pre_election: true
I20260811 05:09:24.320983 11046 raft_consensus.cc:2454] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c8143a5b0414e64b2e909eb58586d83 in term 0.
I20260811 05:09:24.320989 10895 raft_consensus.cc:2454] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c8143a5b0414e64b2e909eb58586d83 in term 0.
I20260811 05:09:24.321476 11120 leader_election.cc:302] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8143a5b0414e64b2e909eb58586d83, 9973381e6f14435b8b75799616446f11; no voters:
I20260811 05:09:24.321830 11268 raft_consensus.cc:2781] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:09:24.321938 11268 raft_consensus.cc:482] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:09:24.322011 11268 raft_consensus.cc:3037] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:24.323136 11268 raft_consensus.cc:504] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.323482 11268 leader_election.cc:288] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [CANDIDATE]: Term 1 election: Requested vote from peers 9973381e6f14435b8b75799616446f11 (127.30.88.194:38257), b068a6a3469d406d885fee44e49d91bb (127.30.88.193:42787)
I20260811 05:09:24.323804 11046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dbdb594594ad6a88519a159e20304" candidate_uuid: "7c8143a5b0414e64b2e909eb58586d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9973381e6f14435b8b75799616446f11"
I20260811 05:09:24.323946 11046 raft_consensus.cc:3037] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:24.323954 10895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dbdb594594ad6a88519a159e20304" candidate_uuid: "7c8143a5b0414e64b2e909eb58586d83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b068a6a3469d406d885fee44e49d91bb"
I20260811 05:09:24.324095 10895 raft_consensus.cc:3037] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:09:24.325352 11046 raft_consensus.cc:2454] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c8143a5b0414e64b2e909eb58586d83 in term 1.
I20260811 05:09:24.325352 10895 raft_consensus.cc:2454] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c8143a5b0414e64b2e909eb58586d83 in term 1.
I20260811 05:09:24.325625 11120 leader_election.cc:302] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8143a5b0414e64b2e909eb58586d83, b068a6a3469d406d885fee44e49d91bb; no voters:
I20260811 05:09:24.325785 11268 raft_consensus.cc:2781] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:09:24.326016 11268 raft_consensus.cc:686] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 1 LEADER]: Becoming Leader. State: Replica: 7c8143a5b0414e64b2e909eb58586d83, State: Running, Role: LEADER
I20260811 05:09:24.326370 11268 consensus_queue.cc:237] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } }
I20260811 05:09:24.329491 10731 catalog_manager.cc:5495] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c8143a5b0414e64b2e909eb58586d83 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "7c8143a5b0414e64b2e909eb58586d83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8143a5b0414e64b2e909eb58586d83" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 } health_report { overall_health: UNKNOWN } } }
I20260811 05:09:24.351205 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.347669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.30.88.195:42643)
0811 05:09:24.347808 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:09:24.347813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:09:24.347828 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:09:24.348041 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:09:24.348044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:09:24.348092 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:09:24.348198 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:09:24.348468 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.348956 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.349761 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:09:24.349961 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:09:24.350093 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:09:24.350408 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:09:24.350776 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:09:24.350943 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 05:09:24.351002 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 05:09:24.351055 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:09:24.356282 11046 raft_consensus.cc:1261] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 1 FOLLOWER]: Refusing update from remote peer 7c8143a5b0414e64b2e909eb58586d83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:24.356283 10895 raft_consensus.cc:1261] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 1 FOLLOWER]: Refusing update from remote peer 7c8143a5b0414e64b2e909eb58586d83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:09:24.356915 11268 consensus_queue.cc:1035] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b068a6a3469d406d885fee44e49d91bb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 42787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:24.357100 11272 consensus_queue.cc:1035] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9973381e6f14435b8b75799616446f11" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:09:24.364847 11278 mvcc.cc:205] Tried to move back new op lower bound from 7317196653997125632 to 7317196653883412480. Current Snapshot: MvccSnapshot[applied={T|T < 7317196653997125632}]
I20260811 05:09:24.372164 10731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:24.372330 10731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:09:24.374799 10731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c013f5040ff6498ead867fe16e2a386f: Sending DeleteTablet for 3 replicas of tablet 3c9dbdb594594ad6a88519a159e20304
I20260811 05:09:24.375550 11171 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dbdb594594ad6a88519a159e20304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55398
I20260811 05:09:24.375629 11024 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dbdb594594ad6a88519a159e20304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:49694
I20260811 05:09:24.375716 10873 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dbdb594594ad6a88519a159e20304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:09:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:35556
I20260811 05:09:24.375998 11287 tablet_replica.cc:330] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: stopping tablet replica
I20260811 05:09:24.376097 11286 tablet_replica.cc:330] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: stopping tablet replica
I20260811 05:09:24.376346 11286 raft_consensus.cc:2229] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:09:24.376385 11288 tablet_replica.cc:330] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb: stopping tablet replica
I20260811 05:09:24.376669 11288 raft_consensus.cc:2229] T 3c9dbdb594594ad6a88519a159e20304 P b068a6a3469d406d885fee44e49d91bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:24.376762 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10805
I20260811 05:09:24.376833 11286 raft_consensus.cc:2258] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:24.382304 11286 ts_tablet_manager.cc:1905] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:24.383311 11287 raft_consensus.cc:2229] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:09:24.383683 11287 raft_consensus.cc:2258] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:09:24.385156 11287 ts_tablet_manager.cc:1905] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:09:24.386004 11286 ts_tablet_manager.cc:1918] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:24.386330 11286 log.cc:1196] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/3c9dbdb594594ad6a88519a159e20304
I20260811 05:09:24.386816 11286 ts_tablet_manager.cc:1939] T 3c9dbdb594594ad6a88519a159e20304 P 7c8143a5b0414e64b2e909eb58586d83: Deleting consensus metadata
I20260811 05:09:24.387557 11287 ts_tablet_manager.cc:1918] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:09:24.387698 11287 log.cc:1196] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/3c9dbdb594594ad6a88519a159e20304
I20260811 05:09:24.388077 11287 ts_tablet_manager.cc:1939] T 3c9dbdb594594ad6a88519a159e20304 P 9973381e6f14435b8b75799616446f11: Deleting consensus metadata
I20260811 05:09:24.389464 10715 catalog_manager.cc:4841] TS 9973381e6f14435b8b75799616446f11 (127.30.88.194:38257): tablet 3c9dbdb594594ad6a88519a159e20304 (table test-table [id=b28a4a4162964d43b7a8acec992bfb92]) successfully deleted
I20260811 05:09:24.389652 10715 catalog_manager.cc:4841] TS 7c8143a5b0414e64b2e909eb58586d83 (127.30.88.195:42643): tablet 3c9dbdb594594ad6a88519a159e20304 (table test-table [id=b28a4a4162964d43b7a8acec992bfb92]) successfully deleted
W20260811 05:09:24.394729 10715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:42787 (error 108)
I20260811 05:09:24.394989 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10956
I20260811 05:09:24.398733 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:09:24.398426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:42787 (local address 127.0.0.1:35570)
0811 05:09:24.398571 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 05:09:24.398646 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:42787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 05:09:24.398933 10715 catalog_manager.cc:4584] TS b068a6a3469d406d885fee44e49d91bb (127.30.88.193:42787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c9dbdb594594ad6a88519a159e20304: Network error: Client connection negotiation failed: client connection to 127.30.88.193:42787: connect: Connection refused (error 111)
I20260811 05:09:24.402796 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 11104
I20260811 05:09:24.410241 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 10699
2026-08-11T05:09:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2288 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:09:24 dist-test-slave-7c35 krb5kdc[11292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:09:24 dist-test-slave-7c35 krb5kdc[11292](info): set up 2 sockets
Aug 11 05:09:24 dist-test-slave-7c35 krb5kdc[11292](info): commencing operation
krb5kdc: starting...
W20260811 05:09:25.463964 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:09:25 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:09:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangeradmin_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangeradmin_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangerlookup_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangerlookup_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
I20260811 05:09:25.627830 31075 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwl2kZi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/postgres -l logfile start
2026-08-11 05:09:27.963 UTC [11330] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:09:27.964 UTC [11330] LOG: listening on IPv4 address "127.30.88.212", port 37675
2026-08-11 05:09:27.967 UTC [11330] LOG: listening on Unix socket "/tmp/.s.PGSQL.37675"
2026-08-11 05:09:27.989 UTC [11333] LOG: database system was shut down at 2026-08-11 05:09:26 UTC
2026-08-11 05:09:27.992 UTC [11330] LOG: database system is ready to accept connections
I20260811 05:09:28.986765 31075 mini_postgres.cc:96] Postgres bound to 37675
2026-08-11 05:09:28.994 UTC [11341] FATAL: database "slave" does not exist
127.30.88.212:37675 - accepting connections
I20260811 05:09:28.995010 31075 mini_ranger.cc:161] Starting Ranger...
I20260811 05:09:29.009850 31075 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:09:29.449290 31075 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:09:29.449391 31075 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin
2026-08-11 05:09:29,720 [I] DB FLAVOR :POSTGRES
2026-08-11 05:09:29,720 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:09:29,720 [I] Checking connection..
2026-08-11 05:09:29,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:09:29,970 [I] Checking connection passed.
2026-08-11 05:09:29,970 [I] --------- Verifying version history table ---------
2026-08-11 05:09:29,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:09:30,209 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:09:30,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:09:30,466 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:09:30,466 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:09:30,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:09:30.714 UTC [11412] WARNING: there is no transaction in progress
2026-08-11 05:09:30,720 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:09:30,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:09:30,961 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:09:30,961 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:09:30,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:09:31,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:09:31,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7c35.c.gcp-upstream.internal','N') ;"
2026-08-11 05:09:31,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:09:31,976 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:09:31,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:09:32,208 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:09:32,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:09:32,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:09:32,734 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:09:32,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:09:33.649 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.664 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.679 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.693 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.707 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.723 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.775 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.781 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.794 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.804 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.813 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.825 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.834 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.840 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.847 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:33.853 UTC [11534] WARNING: there is no transaction in progress
2026-08-11 05:09:34,463 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:09:34,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:09:34,713 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:09:34,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:09:34,949 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:09:34,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:09:35,214 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:09:35,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:09:35,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:09:35,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7c35.c.gcp-upstream.internal';"
2026-08-11 05:09:35,917 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:09:35,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:09:36,151 [I] DB_PATCHES have already been applied
I20260811 05:09:36.157411 31075 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwl2kZi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:09:36.157495 31075 mini_ranger.cc:193] Using host: 127.30.88.212
I20260811 05:09:36.161787 31075 mini_ranger.cc:227] Ranger admin URL: http://127.30.88.212:53443
Aug 11, 2026 5:09:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:09:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:09:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:09:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:09:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:09:37.305444 31075 mini_ranger.cc:160] Time spent starting Ranger: real 8.310s user 0.001s sys 0.010s
Aug 11, 2026 5:09:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:09:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 5:09:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:09:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:09:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:09:58.788618 31075 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangerkms_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/rangerkms_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.88.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.88.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:09:58 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:09:58.875384 31075 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:09:58.885396 31075 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:09:59.679757 31075 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:09:59.679853 31075 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms
I20260811 05:09:59.679960 31075 mini_ranger_kms.cc:227] Using postgres at 127.30.88.212:37675
2026-08-11 05:10:00,308 [I] DB FLAVOR :POSTGRES
2026-08-11 05:10:00,308 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:10:00,308 [I] Checking connection
2026-08-11 05:10:00,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:10:00,577 [I] connection success
2026-08-11 05:10:00,577 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:10:00,577 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:10:00,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:10:00,838 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:10:00,838 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:10:00,838 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:10:00,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:37675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:10:01,079 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:10:01.518543 31075 mini_ranger_kms.cc:310] Created kms service
I20260811 05:10:01.734979 31075 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:10:01.778880 31075 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:10:01 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786424998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:02.167320 31075 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:10:02.167496 31075 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:10:02.167519 31075 mini_ranger_kms.cc:240] Using host: 127.30.88.212
I20260811 05:10:02.172721 31075 mini_ranger_kms.cc:276] Ranger KMS PID: 11737
I20260811 05:10:02.172851 31075 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.88.212:40907
Aug 11, 2026 5:10:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:10:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:10:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:10:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 05:10:04.516685 31075 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.641s user 0.042s sys 0.209s
I20260811 05:10:04.516858 31075 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:10:04.517016 31075 mini_ranger_kms.cc:401] 127.30.88.212:40907/kms/v1/keys
Aug 11 05:10:04 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 05:10:05 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
Aug 11 05:10:05 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
Aug 11 05:10:08 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:10:08.201427 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46623
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:46623
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:40907/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:08.349740 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:08.349977 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:08.350037 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:08.354157 11790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:10:08.354216 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:08.354245 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:08.354264 11790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:10:08.354282 11790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:10:08.359588 11790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.88.212:40907/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:46623
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:08.360486 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:08.361389 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:08.370039 11796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:08.370116 11790 server_base.cc:1034] running on GCE node
W20260811 05:10:08.370067 11799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:08.370067 11797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:08.370922 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:08.371940 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:08.373095 11790 hybrid_clock.cc:648] HybridClock initialized: now 1786425008373069 us; error 37 us; skew 500 ppm
Aug 11 05:10:08 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:08.379814 11790 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:10:08.380686 11790 webserver.cc:460] Webserver started at http://127.30.88.254:34847/ using document root <none> and password file <none>
I20260811 05:10:08.381203 11790 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:08.381258 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:08.381494 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:08 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:08.444900 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "9a00384f124248749b92d3ec86298f72"
format_stamp: "Formatted at 2026-08-11 05:10:08 on dist-test-slave-7c35"
server_key: "b0f70765bdc104f2faa68736f875fdad"
server_key_iv: "b931acc5c25a22044d66bd91757b4e4b"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:08.445653 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "9a00384f124248749b92d3ec86298f72"
format_stamp: "Formatted at 2026-08-11 05:10:08 on dist-test-slave-7c35"
server_key: "b0f70765bdc104f2faa68736f875fdad"
server_key_iv: "b931acc5c25a22044d66bd91757b4e4b"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:08.449858 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 05:10:08.466329 11807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:08.467640 11790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:10:08.467758 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "9a00384f124248749b92d3ec86298f72"
format_stamp: "Formatted at 2026-08-11 05:10:08 on dist-test-slave-7c35"
server_key: "b0f70765bdc104f2faa68736f875fdad"
server_key_iv: "b931acc5c25a22044d66bd91757b4e4b"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:08.467865 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:08.483312 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:08.486052 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:08.486229 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:08.500854 11790 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:46623
I20260811 05:10:08.500897 11868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:46623 every 8 connection(s)
I20260811 05:10:08.501896 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:10:08.504987 11869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:08.509270 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 11790
I20260811 05:10:08.509397 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:10:08.509699 31075 external_mini_cluster.cc:1428] Setting key 9add2d4f97eb2ed8d08cad1cd25fd787
I20260811 05:10:08.511595 11869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: Bootstrap starting.
I20260811 05:10:08.515202 11869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:08.516625 11869 log.cc:824] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:08.518751 11869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: No bootstrap required, opened a new log
Aug 11 05:10:08 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:08.524508 11869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } }
I20260811 05:10:08.524778 11869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:08.524868 11869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a00384f124248749b92d3ec86298f72, State: Initialized, Role: FOLLOWER
I20260811 05:10:08.525385 11869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } }
I20260811 05:10:08.525590 11869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:10:08.525693 11869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:10:08.525818 11869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:08.527192 11869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } }
I20260811 05:10:08.527726 11869 leader_election.cc:302] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a00384f124248749b92d3ec86298f72; no voters:
I20260811 05:10:08.528108 11869 leader_election.cc:288] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:10:08.528216 11876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:10:08.528453 11876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [term 1 LEADER]: Becoming Leader. State: Replica: 9a00384f124248749b92d3ec86298f72, State: Running, Role: LEADER
I20260811 05:10:08.528558 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:08.511284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.30.88.254:46623)
0811 05:10:08.511747 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:08.511786 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:08.511838 (+ 52us) server_negotiation.cc:407] Connection header received
0811 05:10:08.513187 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:08.513233 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:08.513846 (+ 613us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:08.514170 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:08.515173 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:08.517062 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:08.518241 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:08.519006 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:08.519174 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:08.522600 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:08.522607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:08.522633 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:08.522665 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:08.525093 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:08.525661 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:08.525668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:08.525674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:08.525834 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:08.526191 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:08.526195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:08.526196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:08.526774 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:08.527000 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:08.527309 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 05:10:08.527581 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260811 05:10:08.528807 11876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } }
I20260811 05:10:08.529455 11869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:10:08.530139 11876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a00384f124248749b92d3ec86298f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } } }
I20260811 05:10:08.530318 11876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:08.530591 11876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a00384f124248749b92d3ec86298f72. Latest consensus state: current_term: 1 leader_uuid: "9a00384f124248749b92d3ec86298f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a00384f124248749b92d3ec86298f72" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 46623 } } }
I20260811 05:10:08.530731 11876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72 [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:08.532225 11883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:10:08.535434 11883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:10:08.535944 11869 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:10:08.541924 11883 catalog_manager.cc:1347] Generated new cluster ID: 161b77f983d342808042e4b76ffb64ab
I20260811 05:10:08.541982 11883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:10:08.572866 11883 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:10:08.574344 11883 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:10:08.582302 11883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: Generated new TSK 0
I20260811 05:10:08.583299 11883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:10:09.498139 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:40907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:09.665040 11919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:09.665257 11919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:09.665295 11919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:09.669229 11919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:09.669277 11919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:09.669353 11919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:10:09.674993 11919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.88.212:40907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:09.675895 11919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:09.676826 11919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:09.686744 11928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:09.686769 11924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:09.687006 11925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:09.687095 11919 server_base.cc:1034] running on GCE node
I20260811 05:10:09.687578 11919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:09.688371 11919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:09.689556 11919 hybrid_clock.cc:648] HybridClock initialized: now 1786425009689510 us; error 40 us; skew 500 ppm
Aug 11 05:10:09 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:09.693540 11919 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:10:09.694589 11919 webserver.cc:460] Webserver started at http://127.30.88.193:33587/ using document root <none> and password file <none>
I20260811 05:10:09.695235 11919 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:09.695320 11919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:09.695535 11919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:09 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:09.714251 11919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "54dc16d9a8ef4d1e972abe281f7613d4"
format_stamp: "Formatted at 2026-08-11 05:10:09 on dist-test-slave-7c35"
server_key: "ffb215e6c5c1923d3a337070e91c91a5"
server_key_iv: "e6917e8f2383bb45ceaa63a99aee0049"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:09.715180 11919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "54dc16d9a8ef4d1e972abe281f7613d4"
format_stamp: "Formatted at 2026-08-11 05:10:09 on dist-test-slave-7c35"
server_key: "ffb215e6c5c1923d3a337070e91c91a5"
server_key_iv: "e6917e8f2383bb45ceaa63a99aee0049"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:09.720887 11919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:10:09.739450 11937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:09.740867 11919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:10:09.741003 11919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "54dc16d9a8ef4d1e972abe281f7613d4"
format_stamp: "Formatted at 2026-08-11 05:10:09 on dist-test-slave-7c35"
server_key: "ffb215e6c5c1923d3a337070e91c91a5"
server_key_iv: "e6917e8f2383bb45ceaa63a99aee0049"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:09.741128 11919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:09.757813 11919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:09.761654 11919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:09.761831 11919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:09.762578 11919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:09.763706 11919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:09.763757 11919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:09.763795 11919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:09.763810 11919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:09.785633 11919 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:33333
I20260811 05:10:09.785769 12062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:33333 every 8 connection(s)
I20260811 05:10:09.787107 11919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:10:09.795539 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 11919
I20260811 05:10:09.795676 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:10:09.796005 31075 external_mini_cluster.cc:1428] Setting key d5983fccefebb81710195a5ac336bb8f
Aug 11 05:10:09 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:09.805908 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:09.789813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:45459 (local address 127.30.88.254:46623)
0811 05:10:09.790061 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:09.790066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:09.791554 (+ 1488us) server_negotiation.cc:407] Connection header received
0811 05:10:09.792813 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:09.792816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:09.792879 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:09.792978 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:09.794268 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:09.794822 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:09.796584 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:09.796860 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:09.796967 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:09.800487 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:09.800492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:09.800495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:09.800522 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:09.802930 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:09.803721 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:09.803728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:09.803733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:09.803847 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:09.804286 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:09.804290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:09.804292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:09.804628 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:09.804783 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:09.805470 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 05:10:09.805626 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
I20260811 05:10:09.807190 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:09.790563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46623 (local address 127.30.88.193:45459)
0811 05:10:09.791398 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:09.791450 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:09.792577 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:09.793169 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:09.793176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:09.793530 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:09.794118 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:09.794132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:09.794989 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:09.794993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:09.796466 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:09.796475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:09.797000 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:09.797004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:09.797231 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:09.797931 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:09.797952 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:09.800135 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:09.803139 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:09.803146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:09.803171 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:09.803592 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:09.803972 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:09.803977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:09.803982 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:09.804159 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:09.804792 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:09.804800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:09.805151 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 05:10:09.805403 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 05:10:09.805758 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":293,"threads_started":1}
I20260811 05:10:09.808843 12063 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46623
I20260811 05:10:09.809140 12063 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:10:09.809741 12063 heartbeater.cc:511] Master 127.30.88.254:46623 requested a full tablet report, sending...
I20260811 05:10:09.811689 11829 ts_manager.cc:194] Registered new tserver with Master: 54dc16d9a8ef4d1e972abe281f7613d4 (127.30.88.193:33333)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:10:09.813369 11829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:45459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:10:09.829353 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:40907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:10:09 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:10:10.038939 12069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:10.039279 12069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:10.039361 12069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:10.045611 12069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:10.045697 12069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:10.045804 12069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:10:10.054163 12069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.88.212:40907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:10.055735 12069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:10.057210 12069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:10.070878 12077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:10.071012 12078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:10.071408 12081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:10.071442 12069 server_base.cc:1034] running on GCE node
I20260811 05:10:10.072049 12069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:10.072750 12069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:10.073966 12069 hybrid_clock.cc:648] HybridClock initialized: now 1786425010073935 us; error 48 us; skew 500 ppm
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:10.077965 12069 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:10:10.079272 12069 webserver.cc:460] Webserver started at http://127.30.88.194:45075/ using document root <none> and password file <none>
I20260811 05:10:10.080046 12069 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:10.080139 12069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:10.080523 12069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:10.103890 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "20e57a1184bf4673afdae37d1b1824ae"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "d4c5f2d14cec36e2a7697fe76849139b"
server_key_iv: "6178dfa1d3f980c1fc903b7ee7b665bf"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.104920 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "20e57a1184bf4673afdae37d1b1824ae"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "d4c5f2d14cec36e2a7697fe76849139b"
server_key_iv: "6178dfa1d3f980c1fc903b7ee7b665bf"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.112951 12069 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260811 05:10:10.135326 12088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.136888 12069 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.001s
I20260811 05:10:10.137012 12069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "20e57a1184bf4673afdae37d1b1824ae"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "d4c5f2d14cec36e2a7697fe76849139b"
server_key_iv: "6178dfa1d3f980c1fc903b7ee7b665bf"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.137127 12069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:10.165271 12069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:10.171448 12069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:10.171698 12069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:10.172605 12069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:10.174130 12069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:10.174240 12069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.174353 12069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:10.174428 12069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.192557 12069 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:36045
I20260811 05:10:10.193463 12069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:10:10.194155 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 12069
I20260811 05:10:10.194308 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:10:10.194810 31075 external_mini_cluster.cc:1428] Setting key feefd8fb66c61cc88d4355cd426339b1
I20260811 05:10:10.195082 12215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:36045 every 8 connection(s)
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:10.211891 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.197195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:47183 (local address 127.30.88.254:46623)
0811 05:10:10.197340 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.197345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.198710 (+ 1365us) server_negotiation.cc:407] Connection header received
0811 05:10:10.200349 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.200361 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.200435 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.200548 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:10.202345 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.202981 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.204239 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.204490 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.204610 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.207497 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:10.207501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:10.207503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:10.207528 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:10.209220 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.209742 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.209746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.209748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.209830 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.210204 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.210209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.210211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.210392 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:10.210500 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.211553 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.211689 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:10:10.213676 12218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.197833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46623 (local address 127.30.88.194:47183)
0811 05:10:10.198551 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.198599 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.200055 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.200759 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.200771 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.201286 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:10.202108 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.202132 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.203119 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.203122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.204100 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.204110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.204646 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.204650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.204873 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.205561 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:10.205579 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:10.207307 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:10.209344 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:10.209351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:10.209373 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:10.209634 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:10.209938 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:10.209942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:10.209945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:10.210106 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:10.210500 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:10.210506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:10.211248 (+ 742us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.212386 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.212702 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":259,"threads_started":1}
I20260811 05:10:10.215231 12216 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46623
I20260811 05:10:10.215469 12216 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:10.215945 12216 heartbeater.cc:511] Master 127.30.88.254:46623 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:10:10.217028 11829 ts_manager.cc:194] Registered new tserver with Master: 20e57a1184bf4673afdae37d1b1824ae (127.30.88.194:36045)
I20260811 05:10:10.217629 11829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:47183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:10:10.238195 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:40907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:10.412092 12221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:10.412516 12221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:10.412638 12221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:10.420540 12221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:10.420648 12221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:10.420784 12221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:10:10.430569 12221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.88.212:40907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:10.432068 12221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:10.433470 12221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:10.447939 12226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:10.448248 12229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:10.448582 12227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:10.450024 12221 server_base.cc:1034] running on GCE node
I20260811 05:10:10.450695 12221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:10.451552 12221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:10.452735 12221 hybrid_clock.cc:648] HybridClock initialized: now 1786425010452673 us; error 74 us; skew 500 ppm
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:10.457305 12221 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:10:10.458461 12221 webserver.cc:460] Webserver started at http://127.30.88.195:42441/ using document root <none> and password file <none>
I20260811 05:10:10.459194 12221 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:10.459335 12221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:10.459806 12221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:10.477664 12221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "1ba5601dc47f48799ba71b34d41687be"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "bc8ee1fa8a9faee130917341e7fdaa8d"
server_key_iv: "aa346daa3879008738043d1f19cc2ca4"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.478794 12221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "1ba5601dc47f48799ba71b34d41687be"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "bc8ee1fa8a9faee130917341e7fdaa8d"
server_key_iv: "aa346daa3879008738043d1f19cc2ca4"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.498461 12221 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.004s sys 0.000s
I20260811 05:10:10.528445 12236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.530400 12221 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260811 05:10:10.530614 12221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "1ba5601dc47f48799ba71b34d41687be"
format_stamp: "Formatted at 2026-08-11 05:10:10 on dist-test-slave-7c35"
server_key: "bc8ee1fa8a9faee130917341e7fdaa8d"
server_key_iv: "aa346daa3879008738043d1f19cc2ca4"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:10.530891 12221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:10.577127 12221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:10.580969 12221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:10.581238 12221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:10.582124 12221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:10.583505 12221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:10.583626 12221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.583734 12221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:10.583807 12221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:10.607602 12221 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:33663
I20260811 05:10:10.607748 12361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:33663 every 8 connection(s)
I20260811 05:10:10.608881 12221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:10:10.610601 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 12221
I20260811 05:10:10.610714 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:10:10.611037 31075 external_mini_cluster.cc:1428] Setting key 96a4cbd0a0b584cb1abb596bcdd780a7
Aug 11 05:10:10 dist-test-slave-7c35 krb5kdc[11292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:10.630441 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.613054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:41645 (local address 127.30.88.254:46623)
0811 05:10:10.613501 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.613507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.615417 (+ 1910us) server_negotiation.cc:407] Connection header received
0811 05:10:10.616867 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.616870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.616918 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.616989 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:10.618676 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.619495 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.621393 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.621738 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.621832 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.625633 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:10.625637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:10.625640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:10.625667 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:10.627686 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.628442 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.628448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.628450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.628544 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.628994 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.628997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.629000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.629267 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:10.629362 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.630107 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.630239 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
I20260811 05:10:10.632124 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.611752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:46623 (local address 127.30.88.195:41645)
0811 05:10:10.615240 (+ 3488us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.615295 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.616614 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.617294 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.617304 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.617773 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:10.618488 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.618522 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.619665 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.619671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.621207 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.621221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.622062 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.622069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.622392 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.623373 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:10.623397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:10.625453 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:10.627865 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:10.627871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:10.627886 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:10.628295 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:10.628657 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:10.628663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:10.628666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:10.628859 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:10.629459 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:10.629465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:10.629788 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.630631 (+ 843us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.631029 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3248,"thread_start_us":229,"threads_started":1}
I20260811 05:10:10.633447 12362 heartbeater.cc:347] Connected to a master server at 127.30.88.254:46623
I20260811 05:10:10.633699 12362 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:10.634298 12362 heartbeater.cc:511] Master 127.30.88.254:46623 requested a full tablet report, sending...
I20260811 05:10:10.635509 11829 ts_manager.cc:194] Registered new tserver with Master: 1ba5601dc47f48799ba71b34d41687be (127.30.88.195:33663)
I20260811 05:10:10.636219 11829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:41645
I20260811 05:10:10.637570 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:10:10.660495 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.647306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.30.88.254:46623)
0811 05:10:10.647467 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.647471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.651218 (+ 3747us) server_negotiation.cc:407] Connection header received
0811 05:10:10.651447 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.651450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.651505 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.651574 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:10.652587 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.653367 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.654664 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.655051 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.655130 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.657000 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:10.657004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:10.657007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:10.657032 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:10.658666 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.659159 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.659162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.659165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.659255 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:10.659562 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:10.659566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:10.659567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:10.659763 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:10.659855 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.660202 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.660319 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 05:10:10.664407 11829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:10:10.668704 11829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:10:10.701115 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.682595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:36045 (local address 127.0.0.1:48388)
0811 05:10:10.689277 (+ 6682us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.689302 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.689417 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.690133 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.690136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.690171 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:10.691461 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.691470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.693602 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.693606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.696558 (+ 2952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.696570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.700503 (+ 3933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.700507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.700666 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.700706 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.700808 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.700922 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6620,"thread_start_us":97,"threads_started":1}
I20260811 05:10:10.701831 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.682744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.30.88.194:36045)
0811 05:10:10.685809 (+ 3065us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.685863 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.689507 (+ 3644us) server_negotiation.cc:407] Connection header received
0811 05:10:10.689582 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.689587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.689755 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.689943 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:10.691604 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.692913 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.696800 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.700126 (+ 3326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.700238 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.700442 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.701513 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.701621 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2955,"thread_start_us":107,"threads_started":1}
I20260811 05:10:10.702219 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.680923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.30.88.193:33333)
0811 05:10:10.686042 (+ 5119us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.686048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.686070 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:10:10.686142 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.686147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.686336 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.686477 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:10.687794 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.688993 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.700270 (+ 11277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.701230 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.701337 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.701471 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.701943 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.702045 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5015,"thread_start_us":107,"threads_started":1}
I20260811 05:10:10.702389 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.683795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.30.88.195:33663)
0811 05:10:10.693265 (+ 9470us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.693299 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.693324 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:10:10.693413 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.693418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.694989 (+ 1571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.695177 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:10.697295 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.698578 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.700297 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.701216 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.701332 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.701477 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.702094 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.702199 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9357,"thread_start_us":212,"threads_started":1}
I20260811 05:10:10.702500 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.680762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:33333 (local address 127.0.0.1:36266)
0811 05:10:10.685327 (+ 4565us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.685383 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.685501 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.686657 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.686663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.686685 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:10.686800 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.686806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.698916 (+ 12110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.698920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.700114 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.700125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.701534 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.701538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.701619 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.701641 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.701701 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.701761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4355,"thread_start_us":132,"threads_started":1}
I20260811 05:10:10.701989 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.683297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:33663 (local address 127.0.0.1:58520)
0811 05:10:10.691144 (+ 7847us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.691158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.691268 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.695330 (+ 4062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.695333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.695364 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:10.695448 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.695455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.698714 (+ 3259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.698718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.700004 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.700017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.701547 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.701551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.701642 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.701667 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.701745 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.701815 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7783,"thread_start_us":142,"threads_started":1}
I20260811 05:10:10.705861 12143 tablet_service.cc:1467] Processing CreateTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 (DEFAULT_TABLE table=test-table [id=924deb958ff94d39a408e21a3658cd14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:10.706373 12283 tablet_service.cc:1467] Processing CreateTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 (DEFAULT_TABLE table=test-table [id=924deb958ff94d39a408e21a3658cd14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:10.707321 12283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288a9ec8f8104ae79aeee6dc3c676554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:10.707643 12143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288a9ec8f8104ae79aeee6dc3c676554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:10.717880 11989 tablet_service.cc:1467] Processing CreateTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 (DEFAULT_TABLE table=test-table [id=924deb958ff94d39a408e21a3658cd14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:10.719056 11989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288a9ec8f8104ae79aeee6dc3c676554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:10.727584 12381 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Bootstrap starting.
I20260811 05:10:10.731067 12383 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Bootstrap starting.
I20260811 05:10:10.732100 12381 tablet_bootstrap.cc:654] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:10.733402 12381 log.cc:824] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:10.733883 12382 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Bootstrap starting.
I20260811 05:10:10.735647 12381 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: No bootstrap required, opened a new log
I20260811 05:10:10.735829 12381 ts_tablet_manager.cc:1397] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 05:10:10.736485 12383 tablet_bootstrap.cc:654] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:10.738658 12382 tablet_bootstrap.cc:654] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:10.737720 12383 log.cc:824] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:10.740434 12382 log.cc:824] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:10.744614 12383 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: No bootstrap required, opened a new log
I20260811 05:10:10.744355 12381 raft_consensus.cc:348] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.744882 12381 raft_consensus.cc:374] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:10.745000 12381 raft_consensus.cc:729] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba5601dc47f48799ba71b34d41687be, State: Initialized, Role: FOLLOWER
I20260811 05:10:10.745050 12383 ts_tablet_manager.cc:1397] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260811 05:10:10.745918 12381 consensus_queue.cc:260] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.747978 12381 ts_tablet_manager.cc:1428] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260811 05:10:10.748747 12362 heartbeater.cc:503] Master 127.30.88.254:46623 was elected leader, sending a full tablet report...
I20260811 05:10:10.755296 12382 tablet_bootstrap.cc:492] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: No bootstrap required, opened a new log
I20260811 05:10:10.755752 12382 ts_tablet_manager.cc:1397] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.002s
I20260811 05:10:10.764305 12383 raft_consensus.cc:348] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.764711 12383 raft_consensus.cc:374] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:10.764809 12383 raft_consensus.cc:729] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54dc16d9a8ef4d1e972abe281f7613d4, State: Initialized, Role: FOLLOWER
I20260811 05:10:10.765360 12383 consensus_queue.cc:260] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.766673 12383 ts_tablet_manager.cc:1428] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Time spent starting tablet: real 0.021s user 0.000s sys 0.007s
I20260811 05:10:10.767179 12063 heartbeater.cc:503] Master 127.30.88.254:46623 was elected leader, sending a full tablet report...
I20260811 05:10:10.767386 12382 raft_consensus.cc:348] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.767617 12382 raft_consensus.cc:374] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:10.767670 12382 raft_consensus.cc:729] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e57a1184bf4673afdae37d1b1824ae, State: Initialized, Role: FOLLOWER
I20260811 05:10:10.768066 12382 consensus_queue.cc:260] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.769124 12382 ts_tablet_manager.cc:1428] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Time spent starting tablet: real 0.013s user 0.005s sys 0.004s
I20260811 05:10:10.769533 12216 heartbeater.cc:503] Master 127.30.88.254:46623 was elected leader, sending a full tablet report...
W20260811 05:10:10.790405 12064 tablet.cc:2369] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:10:10.841192 12389 raft_consensus.cc:482] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:10:10.841540 12389 raft_consensus.cc:504] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.844321 12389 leader_election.cc:288] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54dc16d9a8ef4d1e972abe281f7613d4 (127.30.88.193:33333), 1ba5601dc47f48799ba71b34d41687be (127.30.88.195:33663)
I20260811 05:10:10.849285 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.843829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:46767 (local address 127.30.88.193:33333)
0811 05:10:10.844036 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.844042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.844719 (+ 677us) server_negotiation.cc:407] Connection header received
0811 05:10:10.844906 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.844911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.844985 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.845069 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:10.845366 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.846184 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.847922 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.848570 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.848671 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.848742 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.849084 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.849156 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 05:10:10.849852 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.843412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:33333 (local address 127.30.88.194:46767)
0811 05:10:10.844617 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.844630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.844778 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.845079 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.845083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.845099 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:10.845227 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.845233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.846362 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.846365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.847797 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.847803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.848838 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.848842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.848945 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.848968 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.849621 (+ 653us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.849699 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1141,"thread_start_us":106,"threads_started":1}
I20260811 05:10:10.851224 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288a9ec8f8104ae79aeee6dc3c676554" candidate_uuid: "20e57a1184bf4673afdae37d1b1824ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" is_pre_election: true
I20260811 05:10:10.851560 12010 raft_consensus.cc:2454] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e57a1184bf4673afdae37d1b1824ae in term 0.
I20260811 05:10:10.852144 12089 leader_election.cc:302] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e57a1184bf4673afdae37d1b1824ae, 54dc16d9a8ef4d1e972abe281f7613d4; no voters:
I20260811 05:10:10.852487 12389 raft_consensus.cc:2781] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:10:10.852576 12389 raft_consensus.cc:482] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:10:10.852629 12389 raft_consensus.cc:3037] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:10.854815 12389 raft_consensus.cc:504] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.855561 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288a9ec8f8104ae79aeee6dc3c676554" candidate_uuid: "20e57a1184bf4673afdae37d1b1824ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54dc16d9a8ef4d1e972abe281f7613d4"
I20260811 05:10:10.855643 12389 leader_election.cc:288] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [CANDIDATE]: Term 1 election: Requested vote from peers 54dc16d9a8ef4d1e972abe281f7613d4 (127.30.88.193:33333), 1ba5601dc47f48799ba71b34d41687be (127.30.88.195:33663)
I20260811 05:10:10.855683 12010 raft_consensus.cc:3037] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:10.856882 12010 raft_consensus.cc:2454] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e57a1184bf4673afdae37d1b1824ae in term 1.
I20260811 05:10:10.857210 12089 leader_election.cc:302] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e57a1184bf4673afdae37d1b1824ae, 54dc16d9a8ef4d1e972abe281f7613d4; no voters:
I20260811 05:10:10.857411 12389 raft_consensus.cc:2781] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:10:10.857627 12389 raft_consensus.cc:686] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 1 LEADER]: Becoming Leader. State: Replica: 20e57a1184bf4673afdae37d1b1824ae, State: Running, Role: LEADER
I20260811 05:10:10.858023 12389 consensus_queue.cc:237] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } }
I20260811 05:10:10.862097 11829 catalog_manager.cc:5495] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 20e57a1184bf4673afdae37d1b1824ae (127.30.88.194). New cstate: current_term: 1 leader_uuid: "20e57a1184bf4673afdae37d1b1824ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20e57a1184bf4673afdae37d1b1824ae" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 36045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 } health_report { overall_health: UNKNOWN } } }
W20260811 05:10:10.867538 12363 tablet.cc:2369] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:10:10.868486 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.844291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:54639 (local address 127.30.88.195:33663)
0811 05:10:10.844423 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.844429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.847345 (+ 2916us) server_negotiation.cc:407] Connection header received
0811 05:10:10.847586 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.847589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.847632 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.847675 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:10.849514 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.850234 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.866404 (+ 16170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.867004 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.867095 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.867142 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.868301 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.868376 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:10:10.869024 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.844067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:33663 (local address 127.30.88.194:54639)
0811 05:10:10.847259 (+ 3192us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.847272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:10.847460 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:10.847799 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:10.847801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:10.847818 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:10.848039 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.848047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.850716 (+ 2669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.850719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.866215 (+ 15496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:10.866230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.867282 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.867286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:10.868156 (+ 870us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.868182 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:10:10.868759 (+ 577us) client_negotiation.cc:240] Negotiation successful
0811 05:10:10.868833 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3117,"thread_start_us":96,"threads_started":1}
I20260811 05:10:10.869936 12301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288a9ec8f8104ae79aeee6dc3c676554" candidate_uuid: "20e57a1184bf4673afdae37d1b1824ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba5601dc47f48799ba71b34d41687be" is_pre_election: true
I20260811 05:10:10.870270 12301 raft_consensus.cc:2454] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e57a1184bf4673afdae37d1b1824ae in term 0.
I20260811 05:10:10.870651 12302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288a9ec8f8104ae79aeee6dc3c676554" candidate_uuid: "20e57a1184bf4673afdae37d1b1824ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba5601dc47f48799ba71b34d41687be"
I20260811 05:10:10.870807 12302 raft_consensus.cc:3037] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:10.872514 12302 raft_consensus.cc:2454] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e57a1184bf4673afdae37d1b1824ae in term 1.
I20260811 05:10:10.883824 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.879264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.30.88.194:36045)
0811 05:10:10.879380 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:10.879384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:10.879395 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:10:10.879460 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:10.879462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:10.879513 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:10.879609 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:10:10.879927 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.880671 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.881854 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:10.882169 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:10.882259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:10.882649 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:10:10.883171 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:10:10.883277 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:10.883657 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 05:10:10.883715 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:10:10.891192 12302 raft_consensus.cc:1261] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 1 FOLLOWER]: Refusing update from remote peer 20e57a1184bf4673afdae37d1b1824ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:10:10.891947 12389 consensus_queue.cc:1035] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba5601dc47f48799ba71b34d41687be" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:10:10.898554 12010 raft_consensus.cc:1261] T 288a9ec8f8104ae79aeee6dc3c676554 P 54dc16d9a8ef4d1e972abe281f7613d4 [term 1 FOLLOWER]: Refusing update from remote peer 20e57a1184bf4673afdae37d1b1824ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:10:10.899204 12389 consensus_queue.cc:1035] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "54dc16d9a8ef4d1e972abe281f7613d4" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:10:10.916889 12395 mvcc.cc:205] Tried to move back new op lower bound from 7317196844601962496 to 7317196844478607360. Current Snapshot: MvccSnapshot[applied={T|T < 7317196844601962496}]
I20260811 05:10:10.921614 12398 mvcc.cc:205] Tried to move back new op lower bound from 7317196844601962496 to 7317196844478607360. Current Snapshot: MvccSnapshot[applied={T|T < 7317196844601962496}]
I20260811 05:10:10.922087 12396 mvcc.cc:205] Tried to move back new op lower bound from 7317196844601962496 to 7317196844478607360. Current Snapshot: MvccSnapshot[applied={T|T < 7317196844601962496}]
I20260811 05:10:10.935158 11829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:10:10.935317 11829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:10:10.937897 11829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a00384f124248749b92d3ec86298f72: Sending DeleteTablet for 3 replicas of tablet 288a9ec8f8104ae79aeee6dc3c676554
I20260811 05:10:10.938719 11989 tablet_service.cc:1514] Processing DeleteTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:10 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:36266
I20260811 05:10:10.939060 12283 tablet_service.cc:1514] Processing DeleteTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:10 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:58520
I20260811 05:10:10.939205 12143 tablet_service.cc:1514] Processing DeleteTablet for tablet 288a9ec8f8104ae79aeee6dc3c676554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:10 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:48388
I20260811 05:10:10.939924 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 11919
I20260811 05:10:10.959391 12406 tablet_replica.cc:330] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: stopping tablet replica
I20260811 05:10:10.959789 12406 raft_consensus.cc:2229] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:10:10.960330 12406 raft_consensus.cc:2258] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:10:10.963577 12405 tablet_replica.cc:330] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: stopping tablet replica
I20260811 05:10:10.965042 12406 ts_tablet_manager.cc:1905] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:10:10.965421 12405 raft_consensus.cc:2229] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:10:10.965817 12405 raft_consensus.cc:2258] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:10:10.968828 12406 ts_tablet_manager.cc:1918] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:10:10.972168 12405 ts_tablet_manager.cc:1905] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:10:10.974933 12406 log.cc:1196] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/288a9ec8f8104ae79aeee6dc3c676554
I20260811 05:10:10.977943 12406 ts_tablet_manager.cc:1939] T 288a9ec8f8104ae79aeee6dc3c676554 P 20e57a1184bf4673afdae37d1b1824ae: Deleting consensus metadata
I20260811 05:10:10.979287 11810 catalog_manager.cc:4841] TS 20e57a1184bf4673afdae37d1b1824ae (127.30.88.194:36045): tablet 288a9ec8f8104ae79aeee6dc3c676554 (table test-table [id=924deb958ff94d39a408e21a3658cd14]) successfully deleted
I20260811 05:10:10.974813 12405 ts_tablet_manager.cc:1918] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:10:10.979628 12405 log.cc:1196] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/288a9ec8f8104ae79aeee6dc3c676554
I20260811 05:10:10.987615 12405 ts_tablet_manager.cc:1939] T 288a9ec8f8104ae79aeee6dc3c676554 P 1ba5601dc47f48799ba71b34d41687be: Deleting consensus metadata
I20260811 05:10:10.989086 11810 catalog_manager.cc:4841] TS 1ba5601dc47f48799ba71b34d41687be (127.30.88.195:33663): tablet 288a9ec8f8104ae79aeee6dc3c676554 (table test-table [id=924deb958ff94d39a408e21a3658cd14]) successfully deleted
W20260811 05:10:10.989562 11809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:33333 (error 108)
I20260811 05:10:10.990633 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 12069
I20260811 05:10:10.998523 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:10.998260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:33333 (local address 127.0.0.1:36278)
0811 05:10:10.998374 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:10.998450 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 05:10:10.998739 11809 catalog_manager.cc:4584] TS 54dc16d9a8ef4d1e972abe281f7613d4 (127.30.88.193:33333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 288a9ec8f8104ae79aeee6dc3c676554: Network error: Client connection negotiation failed: client connection to 127.30.88.193:33333: connect: Connection refused (error 111)
I20260811 05:10:11.001209 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 12221
I20260811 05:10:11.019893 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 11790
I20260811 05:10:11.030210 31075 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:10:11.375116 31075 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:10:11.375237 31075 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:10:11.720244 31075 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:10:11.720 UTC [11330] LOG: received smart shutdown request
2026-08-11 05:10:11.724 UTC [11330] LOG: background worker "logical replication launcher" (PID 11339) exited with exit code 1
2026-08-11 05:10:11.725 UTC [11334] LOG: shutting down
2026-08-11 05:10:11.882 UTC [11330] LOG: database system is shut down
2026-08-11T05:10:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47595 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:10:12 dist-test-slave-7c35 krb5kdc[12418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:10:12 dist-test-slave-7c35 krb5kdc[12418](info): set up 2 sockets
Aug 11 05:10:12 dist-test-slave-7c35 krb5kdc[12418](info): commencing operation
krb5kdc: starting...
W20260811 05:10:13.107486 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:10:13 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:10:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangeradmin_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangeradmin_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangerlookup_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangerlookup_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
I20260811 05:10:13.267666 31075 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwl2kZi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/postgres -l logfile start
2026-08-11 05:10:15.609 UTC [12453] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:10:15.609 UTC [12453] LOG: listening on IPv4 address "127.30.88.212", port 55509
2026-08-11 05:10:15.612 UTC [12453] LOG: listening on Unix socket "/tmp/.s.PGSQL.55509"
2026-08-11 05:10:15.636 UTC [12456] LOG: database system was shut down at 2026-08-11 05:10:14 UTC
2026-08-11 05:10:15.640 UTC [12453] LOG: database system is ready to accept connections
I20260811 05:10:16.658715 31075 mini_postgres.cc:96] Postgres bound to 55509
2026-08-11 05:10:16.666 UTC [12464] FATAL: database "slave" does not exist
127.30.88.212:55509 - accepting connections
I20260811 05:10:16.666718 31075 mini_ranger.cc:161] Starting Ranger...
I20260811 05:10:16.682031 31075 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:10:17.116416 31075 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:10:17.116508 31075 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin
2026-08-11 05:10:17,391 [I] DB FLAVOR :POSTGRES
2026-08-11 05:10:17,391 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:10:17,391 [I] Checking connection..
2026-08-11 05:10:17,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:10:17,655 [I] Checking connection passed.
2026-08-11 05:10:17,655 [I] --------- Verifying version history table ---------
2026-08-11 05:10:17,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:10:17,906 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:10:17,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:10:18,170 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:10:18,170 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:10:18,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:10:18.411 UTC [12535] WARNING: there is no transaction in progress
2026-08-11 05:10:18,419 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:10:18,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:10:18,679 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:10:18,679 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:10:18,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:10:18,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:10:19,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7c35.c.gcp-upstream.internal','N') ;"
2026-08-11 05:10:19,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:10:19,663 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:10:19,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:10:19,925 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:10:19,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:10:20,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:10:20,402 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:10:20,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:10:21.345 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.361 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.376 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.390 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.406 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.422 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.472 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.479 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.491 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.501 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.511 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.524 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.533 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.539 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.546 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:21.552 UTC [12653] WARNING: there is no transaction in progress
2026-08-11 05:10:22,149 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:10:22,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:10:22,374 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:10:22,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:10:22,633 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:10:22,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:10:22,871 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:10:22,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:10:23,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:10:23,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7c35.c.gcp-upstream.internal';"
2026-08-11 05:10:23,620 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:10:23,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:10:23,875 [I] DB_PATCHES have already been applied
I20260811 05:10:23.881299 31075 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwl2kZi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:10:23.881364 31075 mini_ranger.cc:193] Using host: 127.30.88.212
I20260811 05:10:23.886031 31075 mini_ranger.cc:227] Ranger admin URL: http://127.30.88.212:56415
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:10:25.072683 31075 mini_ranger.cc:160] Time spent starting Ranger: real 8.406s user 0.000s sys 0.012s
Aug 11, 2026 5:10:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:10:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 5:10:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:10:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:10:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:10:45.766677 31075 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.88.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.88.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangerkms_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.88.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/rangerkms_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.88.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.88.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.88.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/HTTP_127.30.88.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:10:45 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425045, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:10:45.845100 31075 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:10:45.854111 31075 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:10:46.649214 31075 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:10:46.649298 31075 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms
I20260811 05:10:46.649319 31075 mini_ranger_kms.cc:227] Using postgres at 127.30.88.212:55509
2026-08-11 05:10:46,828 [I] DB FLAVOR :POSTGRES
2026-08-11 05:10:46,828 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:10:46,829 [I] Checking connection
2026-08-11 05:10:46,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:10:47,077 [I] connection success
2026-08-11 05:10:47,078 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:10:47,078 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:10:47,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:10:47,308 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:10:47,309 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:10:47,309 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:10:47,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.88.212:55509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:10:47,543 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:10:47.952685 31075 mini_ranger_kms.cc:310] Created kms service
I20260811 05:10:48.144457 31075 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:10:48.188673 31075 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:10:48 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425045, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:48.584000 31075 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:10:48.584177 31075 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwl2kZi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwl2kZi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:10:48.584198 31075 mini_ranger_kms.cc:240] Using host: 127.30.88.212
I20260811 05:10:48.588737 31075 mini_ranger_kms.cc:276] Ranger KMS PID: 12851
I20260811 05:10:48.588820 31075 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.88.212:44849
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 05:10:50 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:50.940366 31075 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.095s user 0.001s sys 0.164s
I20260811 05:10:50.940492 31075 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:10:50.940558 31075 mini_ranger_kms.cc:401] 127.30.88.212:44849/kms/v1/keys
Aug 11 05:10:51 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
Aug 11 05:10:51 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.88.212@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
Aug 11 05:10:53 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:10:53.724054 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43509
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43509
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:53.847365 12905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:53.847620 12905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:53.847658 12905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:53.851446 12905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:10:53.851490 12905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:53.851507 12905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:53.851521 12905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:10:53.851534 12905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:10:53.856513 12905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.88.212:44849/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43509
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:53.857458 12905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:53.858494 12905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:53.866971 12913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:53.866978 12910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:53.867007 12911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:53.867205 12905 server_base.cc:1034] running on GCE node
I20260811 05:10:53.867846 12905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:53.868889 12905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:53.870064 12905 hybrid_clock.cc:648] HybridClock initialized: now 1786425053870050 us; error 33 us; skew 500 ppm
Aug 11 05:10:53 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425053, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:53.873009 12905 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:10:53.874053 12905 webserver.cc:460] Webserver started at http://127.30.88.254:38391/ using document root <none> and password file <none>
I20260811 05:10:53.874696 12905 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:53.874780 12905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:53.875089 12905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:53 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425053, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:53.937541 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "04832409d53d4634b7497024c2affd2d"
format_stamp: "Formatted at 2026-08-11 05:10:53 on dist-test-slave-7c35"
server_key: "2dc3709fd896df9e59f7086f83c6f6a9"
server_key_iv: "d99e4685284f48f869f0e92bb8dde7aa"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:53.938373 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "04832409d53d4634b7497024c2affd2d"
format_stamp: "Formatted at 2026-08-11 05:10:53 on dist-test-slave-7c35"
server_key: "2dc3709fd896df9e59f7086f83c6f6a9"
server_key_iv: "d99e4685284f48f869f0e92bb8dde7aa"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:53.943575 12905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 05:10:53.958662 12920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:53.960017 12905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:10:53.960156 12905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "04832409d53d4634b7497024c2affd2d"
format_stamp: "Formatted at 2026-08-11 05:10:53 on dist-test-slave-7c35"
server_key: "2dc3709fd896df9e59f7086f83c6f6a9"
server_key_iv: "d99e4685284f48f869f0e92bb8dde7aa"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:53.960294 12905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:53.973857 12905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:53.976858 12905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:53.977061 12905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:53.985514 12905 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:43509
I20260811 05:10:53.985520 12976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:43509 every 8 connection(s)
I20260811 05:10:53.986575 12905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:10:53.989491 12977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:53.991529 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 12905
I20260811 05:10:53.991667 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:10:53.991966 31075 external_mini_cluster.cc:1428] Setting key 07e95ab5f2bcf5b473dd2245a9ecdc83
I20260811 05:10:53.996572 12977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Bootstrap starting.
I20260811 05:10:53.999028 12977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:53.999831 12977 log.cc:824] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:54.002100 12977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: No bootstrap required, opened a new log
Aug 11 05:10:53 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:54.006232 12977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:54.006575 12977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:54.006709 12977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04832409d53d4634b7497024c2affd2d, State: Initialized, Role: FOLLOWER
I20260811 05:10:54.007287 12977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:54.007485 12977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:10:54.007611 12977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:10:54.007727 12977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:54.007858 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:53.993482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.30.88.254:43509)
0811 05:10:53.993983 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:53.994005 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:53.994054 (+ 49us) server_negotiation.cc:407] Connection header received
0811 05:10:53.994956 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:53.994980 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:53.995370 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:53.995717 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:53.996538 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:53.998116 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:53.999446 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:54.000209 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:54.000387 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:54.003389 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:54.003397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:54.003411 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:54.003439 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:54.005111 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:54.005539 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:54.005544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:54.005548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:54.005652 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:54.005911 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:54.005915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:54.005916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:54.006265 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:54.006471 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:54.006676 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 05:10:54.006978 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":136,"threads_started":1}
I20260811 05:10:54.008912 12977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:54.009281 12977 leader_election.cc:302] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04832409d53d4634b7497024c2affd2d; no voters:
I20260811 05:10:54.009634 12977 leader_election.cc:288] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:10:54.009773 12983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:10:54.010028 12983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 LEADER]: Becoming Leader. State: Replica: 04832409d53d4634b7497024c2affd2d, State: Running, Role: LEADER
I20260811 05:10:54.010427 12983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:54.011081 12977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:10:54.011591 12983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04832409d53d4634b7497024c2affd2d. Latest consensus state: current_term: 1 leader_uuid: "04832409d53d4634b7497024c2affd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } } }
I20260811 05:10:54.011775 12983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:54.012223 12990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:10:54.012611 12985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04832409d53d4634b7497024c2affd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } } }
I20260811 05:10:54.012730 12985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:54.015328 12990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:10:54.015729 12977 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:10:54.027127 12990 catalog_manager.cc:1347] Generated new cluster ID: 79acd59b94504901b07552782f2e34cb
I20260811 05:10:54.027201 12990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:10:54.039328 12990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:10:54.040625 12990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:10:54.048703 12990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Generated new TSK 0
I20260811 05:10:54.049547 12990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:10:54.781515 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:54.900566 13027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:54.900812 13027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:54.900887 13027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:54.905140 13027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:54.905217 13027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:54.905298 13027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:10:54.910383 13027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:54.911464 13027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:54.912566 13027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:54.922204 13038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:54.922224 13035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:54.922204 13036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:54.923187 13027 server_base.cc:1034] running on GCE node
I20260811 05:10:54.923564 13027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:54.924193 13027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:54.925369 13027 hybrid_clock.cc:648] HybridClock initialized: now 1786425054925349 us; error 35 us; skew 500 ppm
Aug 11 05:10:54 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:54.928359 13027 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:10:54.929224 13027 webserver.cc:460] Webserver started at http://127.30.88.193:44655/ using document root <none> and password file <none>
I20260811 05:10:54.929754 13027 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:54.929812 13027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:54.930012 13027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:54 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:54.946482 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "5f5f5212b2e043ca818d916a253660e0"
format_stamp: "Formatted at 2026-08-11 05:10:54 on dist-test-slave-7c35"
server_key: "b98e5f32fbf41bdd1d33fa9d9aaaf375"
server_key_iv: "1a08087c55aed1f3bbad8b17687b0462"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:54.947722 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "5f5f5212b2e043ca818d916a253660e0"
format_stamp: "Formatted at 2026-08-11 05:10:54 on dist-test-slave-7c35"
server_key: "b98e5f32fbf41bdd1d33fa9d9aaaf375"
server_key_iv: "1a08087c55aed1f3bbad8b17687b0462"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:54.954149 13027 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260811 05:10:54.970371 13048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:54.972079 13027 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 05:10:54.972276 13027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "5f5f5212b2e043ca818d916a253660e0"
format_stamp: "Formatted at 2026-08-11 05:10:54 on dist-test-slave-7c35"
server_key: "b98e5f32fbf41bdd1d33fa9d9aaaf375"
server_key_iv: "1a08087c55aed1f3bbad8b17687b0462"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:54.972515 13027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:55.040112 13027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:55.043681 13027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:55.043920 13027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:55.044592 13027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:55.045578 13027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:55.045683 13027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.045791 13027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:55.045861 13027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.058530 13027 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:46865
I20260811 05:10:55.059634 13027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:10:55.061314 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13027
I20260811 05:10:55.061432 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:10:55.061717 31075 external_mini_cluster.cc:1428] Setting key 93a47518d1de31f73719d0b7b080d95f
I20260811 05:10:55.060488 13172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:46865 every 8 connection(s)
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
I20260811 05:10:55.092597 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.063650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.193:34945)
0811 05:10:55.067947 (+ 4297us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:55.067999 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:55.068990 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:55.073225 (+ 4235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:55.073234 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:55.073550 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:55.074152 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.074170 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.075369 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.075373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.076708 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.076719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.078786 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.078790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.079025 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.081966 (+ 2941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:55.081990 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:55.086373 (+ 4383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:55.088826 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.088833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.088850 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.089206 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.089601 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.089605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.089607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.089802 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.090564 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:55.090571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:55.090936 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 05:10:55.091137 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 05:10:55.091536 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4082,"thread_start_us":147,"threads_started":1}
I20260811 05:10:55.094120 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.064633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:34945 (local address 127.30.88.254:43509)
0811 05:10:55.069389 (+ 4756us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:55.069393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:55.069406 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:10:55.069445 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:55.069448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:55.069491 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:55.069570 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:55.074336 (+ 4766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.075225 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.076842 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.077188 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.077278 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.086625 (+ 9347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:55.086629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:55.086631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:55.086661 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:55.088664 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.089377 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.089380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.089382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.089481 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.089931 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.089934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.089936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.090188 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:55.093747 (+ 3559us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:55.093825 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 05:10:55.093953 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4685,"thread_start_us":81,"threads_started":1}
I20260811 05:10:55.095675 13173 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:55.095932 13173 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:55.096513 13173 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:55.098227 12937 ts_manager.cc:194] Registered new tserver with Master: 5f5f5212b2e043ca818d916a253660e0 (127.30.88.193:46865)
I20260811 05:10:55.099723 12937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:34945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:10:55.115208 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:10:55.274634 13181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:55.274976 13181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:55.275040 13181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:55.281152 13181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:55.281237 13181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:55.281319 13181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:10:55.289116 13181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:55.290501 13181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:55.291874 13181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:55.304472 13188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:55.304725 13189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:55.304975 13181 server_base.cc:1034] running on GCE node
W20260811 05:10:55.305466 13191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:55.305959 13181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:55.306743 13181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:55.307940 13181 hybrid_clock.cc:648] HybridClock initialized: now 1786425055307916 us; error 35 us; skew 500 ppm
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:55.311916 13181 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:10:55.313144 13181 webserver.cc:460] Webserver started at http://127.30.88.194:42111/ using document root <none> and password file <none>
I20260811 05:10:55.313936 13181 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:55.314039 13181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:55.314308 13181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:55.329528 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "a3f4c170105548cea5ef503328ba0ad8"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "bcc2710e6c438a8a6c5f76340c277acc"
server_key_iv: "63b0f38cf5b5cc54a1d0e87613fbc78d"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.330338 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "a3f4c170105548cea5ef503328ba0ad8"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "bcc2710e6c438a8a6c5f76340c277acc"
server_key_iv: "63b0f38cf5b5cc54a1d0e87613fbc78d"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.335533 13181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:10:55.356380 13198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.360031 13181 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:10:55.360407 13181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "a3f4c170105548cea5ef503328ba0ad8"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "bcc2710e6c438a8a6c5f76340c277acc"
server_key_iv: "63b0f38cf5b5cc54a1d0e87613fbc78d"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.360555 13181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:55.399353 13181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:55.403155 13181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:55.403421 13181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:55.404132 13181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:55.405215 13181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:55.405318 13181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.405438 13181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:55.405499 13181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.419168 13181 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:33723
I20260811 05:10:55.420315 13181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:10:55.421258 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13181
I20260811 05:10:55.421375 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:10:55.421684 31075 external_mini_cluster.cc:1428] Setting key 96e85b244669a0a046755c1e260d50e6
I20260811 05:10:55.427919 13329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:33723 every 8 connection(s)
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:10:55.458254 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.430074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:58193 (local address 127.30.88.254:43509)
0811 05:10:55.430247 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:55.430252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:55.433934 (+ 3682us) server_negotiation.cc:407] Connection header received
0811 05:10:55.435092 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:55.435095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:55.435143 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:55.435247 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:55.436596 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.437124 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.438929 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.439147 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.439261 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.453229 (+ 13968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:55.453234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:55.453237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:55.453266 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:55.455482 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.456234 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.456238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.456239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.456323 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.456788 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.456792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.456793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.457044 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:55.457154 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:55.457924 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 05:10:55.458078 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:10:55.458765 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:10:55.459890 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.430684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.194:58193)
0811 05:10:55.433788 (+ 3104us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:55.433841 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:55.434890 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:55.435448 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:55.435459 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:55.435770 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:55.436456 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.436478 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.437310 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.437313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.438783 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.438795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.439297 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.439301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.439492 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.451104 (+ 11612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:55.451133 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:55.452997 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:55.455756 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.455762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.455780 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.456114 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.456445 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.456448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.456450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.456622 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.457218 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:55.457226 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:55.457607 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 05:10:55.458453 (+ 846us) client_negotiation.cc:240] Negotiation successful
0811 05:10:55.458810 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2880,"thread_start_us":171,"threads_started":1}
I20260811 05:10:55.461278 13330 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:55.461531 13330 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:55.462122 13330 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:55.463387 12937 ts_manager.cc:194] Registered new tserver with Master: a3f4c170105548cea5ef503328ba0ad8 (127.30.88.194:33723)
I20260811 05:10:55.464125 12937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:58193
W20260811 05:10:55.598695 13335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:55.599133 13335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:55.599258 13335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:55.603282 13335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:55.603356 13335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:55.603475 13335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:10:55.608980 13335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:55.610219 13335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:55.611624 13335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:55.627471 13341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:55.627914 13340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:55.628262 13344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:55.631378 13335 server_base.cc:1034] running on GCE node
I20260811 05:10:55.631850 13335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:55.632520 13335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:55.633740 13335 hybrid_clock.cc:648] HybridClock initialized: now 1786425055633708 us; error 49 us; skew 500 ppm
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:55.637401 13335 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:10:55.638531 13335 webserver.cc:460] Webserver started at http://127.30.88.195:36053/ using document root <none> and password file <none>
I20260811 05:10:55.639252 13335 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:55.639350 13335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:10:55.639591 13335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:55.660118 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "b3630674701048ada1f4bdbe1b091584"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "00e1e918dfdfd6cb146c850496eba0f2"
server_key_iv: "60186cf1a09df9d35f7fd8dde8f93e93"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.660948 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "b3630674701048ada1f4bdbe1b091584"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "00e1e918dfdfd6cb146c850496eba0f2"
server_key_iv: "60186cf1a09df9d35f7fd8dde8f93e93"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.666043 13335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:55.711226 13352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.712973 13335 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.000s
I20260811 05:10:55.713126 13335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "b3630674701048ada1f4bdbe1b091584"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "00e1e918dfdfd6cb146c850496eba0f2"
server_key_iv: "60186cf1a09df9d35f7fd8dde8f93e93"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:55.713265 13335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:55.745417 13335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:55.749387 13335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:55.749619 13335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:55.750351 13335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:55.751665 13335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:55.751765 13335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.751852 13335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:55.751906 13335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:55.765352 13335 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:33189
I20260811 05:10:55.766506 13335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:10:55.767100 13480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:33189 every 8 connection(s)
I20260811 05:10:55.773157 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13335
I20260811 05:10:55.773314 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:10:55.773629 31075 external_mini_cluster.cc:1428] Setting key 2acbc332f5f5fce13e46af2ebcc18ad8
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:55.784269 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.769545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:57191 (local address 127.30.88.254:43509)
0811 05:10:55.769681 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:55.769685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:55.771066 (+ 1381us) server_negotiation.cc:407] Connection header received
0811 05:10:55.772291 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:55.772294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:55.772347 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:55.772433 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:55.773910 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.774780 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.776465 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.776814 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.776891 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.780045 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:55.780049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:55.780059 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:55.780086 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:55.781817 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.782489 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.782493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.782495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.782585 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:55.783033 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:55.783036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:55.783038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:55.783268 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:55.783364 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:55.783973 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 05:10:55.784106 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:10:55.785312 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:55.770286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.195:57191)
0811 05:10:55.770907 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:55.770965 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:55.772040 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:55.772632 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:55.772641 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:55.773027 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:55.773748 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.773770 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.774963 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.774966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.776323 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:55.776334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:55.777097 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:55.777100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:55.777291 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:55.777997 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:55.778017 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:55.779845 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:55.781957 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.781963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.781979 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.782348 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.782688 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:55.782692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:55.782694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:55.782920 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:55.783386 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:55.783392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:55.783695 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 05:10:55.783901 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 05:10:55.784220 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":161,"threads_started":1}
I20260811 05:10:55.786671 13481 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:55.786993 13481 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:55.787556 13481 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:55.788846 12937 ts_manager.cc:194] Registered new tserver with Master: b3630674701048ada1f4bdbe1b091584 (127.30.88.195:33189)
I20260811 05:10:55.789566 12937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:57191
I20260811 05:10:55.800632 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:10:55.801396 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13027
I20260811 05:10:55.817090 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13181
I20260811 05:10:55.826069 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13335
I20260811 05:10:55.834976 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 12905
I20260811 05:10:55.843262 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43509
--webserver_interface=127.30.88.254
--webserver_port=38391
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43509
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:10:55.949702 13484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:55.949949 13484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:55.950017 13484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:55.954326 13484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:10:55.954385 13484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:55.954408 13484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:55.954432 13484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:10:55.954490 13484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:10:55.960326 13484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43509
--ranger_config_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=38391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:55.961400 13484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:55.962298 13484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:55.970566 13491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:55.970630 13484 server_base.cc:1034] running on GCE node
W20260811 05:10:55.970603 13493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:55.970780 13489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:55.971310 13484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:55.972301 13484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:55.973477 13484 hybrid_clock.cc:648] HybridClock initialized: now 1786425055973459 us; error 30 us; skew 500 ppm
Aug 11 05:10:55 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:55.976155 13484 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:10:55.976953 13484 webserver.cc:460] Webserver started at http://127.30.88.254:38391/ using document root <none> and password file <none>
I20260811 05:10:55.977509 13484 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:55.977622 13484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:56.048789 13484 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:10:56.051369 13501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.052655 13484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:10:56.052843 13484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "04832409d53d4634b7497024c2affd2d"
format_stamp: "Formatted at 2026-08-11 05:10:53 on dist-test-slave-7c35"
server_key: "2dc3709fd896df9e59f7086f83c6f6a9"
server_key_iv: "d99e4685284f48f869f0e92bb8dde7aa"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:56.053385 13484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:56.062960 13484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:56.065165 13484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:56.065369 13484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:56.072297 13484 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:43509
I20260811 05:10:56.072304 13562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:43509 every 8 connection(s)
I20260811 05:10:56.073263 13484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:10:56.077237 13563 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:10:56.078783 13563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Bootstrap starting.
I20260811 05:10:56.080484 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13484
I20260811 05:10:56.080991 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:46865
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:43509
--webserver_port=44655
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:10:56.095734 13563 log.cc:824] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:56.101969 13563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:10:56.102349 13563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Bootstrap complete.
I20260811 05:10:56.106422 13563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:56.106791 13563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04832409d53d4634b7497024c2affd2d, State: Initialized, Role: FOLLOWER
I20260811 05:10:56.107277 13563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:56.107385 13563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:10:56.107470 13563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:10:56.107573 13563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:10:56.109356 13563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:56.109678 13563 leader_election.cc:302] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04832409d53d4634b7497024c2affd2d; no voters:
I20260811 05:10:56.109951 13563 leader_election.cc:288] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:10:56.110111 13567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:10:56.110348 13567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [term 2 LEADER]: Becoming Leader. State: Replica: 04832409d53d4634b7497024c2affd2d, State: Running, Role: LEADER
I20260811 05:10:56.110705 13567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } }
I20260811 05:10:56.111171 13563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:10:56.112205 13568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "04832409d53d4634b7497024c2affd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } } }
I20260811 05:10:56.112352 13568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:56.113157 13577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:10:56.113498 13563 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:10:56.113137 13569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04832409d53d4634b7497024c2affd2d. Latest consensus state: current_term: 2 leader_uuid: "04832409d53d4634b7497024c2affd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04832409d53d4634b7497024c2affd2d" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43509 } } }
I20260811 05:10:56.114323 13569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d [sys.catalog]: This master's current role is: LEADER
I20260811 05:10:56.118748 13577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:10:56.119819 13577 catalog_manager.cc:1259] Loaded cluster ID: 79acd59b94504901b07552782f2e34cb
I20260811 05:10:56.119925 13577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:10:56.122900 13577 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:10:56.123805 13577 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Loaded TSK: 0
I20260811 05:10:56.125150 13577 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:10:56.223070 13565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:56.223415 13565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:56.223497 13565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:56.229255 13565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:56.229358 13565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:56.229480 13565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:10:56.236582 13565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:46865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=44655
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:56.237699 13565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:56.238932 13565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:56.250483 13603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:56.251183 13602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:56.251713 13605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:56.252300 13565 server_base.cc:1034] running on GCE node
I20260811 05:10:56.252732 13565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:56.253489 13565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:56.256748 13565 hybrid_clock.cc:648] HybridClock initialized: now 1786425056256727 us; error 31 us; skew 500 ppm
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:56.264434 13565 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:10:56.265580 13565 webserver.cc:460] Webserver started at http://127.30.88.193:44655/ using document root <none> and password file <none>
I20260811 05:10:56.266315 13565 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:56.266484 13565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:56.291772 13565 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 05:10:56.295712 13614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.297328 13565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 05:10:56.297449 13565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "5f5f5212b2e043ca818d916a253660e0"
format_stamp: "Formatted at 2026-08-11 05:10:54 on dist-test-slave-7c35"
server_key: "b98e5f32fbf41bdd1d33fa9d9aaaf375"
server_key_iv: "1a08087c55aed1f3bbad8b17687b0462"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:56.298158 13565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:56.316485 13565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:56.320101 13565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:56.320461 13565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:56.321169 13565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:56.322234 13565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:56.322337 13565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.322469 13565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:56.322558 13565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.333710 13565 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:46865
I20260811 05:10:56.333722 13742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:46865 every 8 connection(s)
I20260811 05:10:56.336184 13565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:10:56.340863 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13565
I20260811 05:10:56.341353 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:33723
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:43509
--webserver_port=42111
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:56.368140 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.341283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.193:38225)
0811 05:10:56.341758 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:56.341797 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:56.342680 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:56.350472 (+ 7792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:56.350486 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:56.350928 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:56.351700 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.351726 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.354174 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.354181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.355762 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.355776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.357066 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.357071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.357282 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.358058 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:56.358078 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:56.359784 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:56.364675 (+ 4891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.364684 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.364696 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.365028 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.365473 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.365476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.365478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.365637 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.366386 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:56.366393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:56.366674 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 05:10:56.366891 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 05:10:56.367186 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":143,"threads_started":1}
I20260811 05:10:56.369881 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.341156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38225 (local address 127.30.88.254:43509)
0811 05:10:56.348998 (+ 7842us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:56.349024 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:56.349081 (+ 57us) server_negotiation.cc:407] Connection header received
0811 05:10:56.349421 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:56.349441 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:56.349915 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:56.351990 (+ 2075us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:56.352329 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.354003 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.355948 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.356870 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.357833 (+ 963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.359993 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:56.360000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:56.360014 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:56.360041 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:56.364545 (+ 4504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.365202 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.365209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.365214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.365364 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.365764 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.365768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.365770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.366261 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:56.366404 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:56.368657 (+ 2253us) server_negotiation.cc:299] Negotiation successful
0811 05:10:56.369027 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7633,"thread_start_us":107,"threads_started":1}
I20260811 05:10:56.372228 13743 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:56.372486 13743 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.373032 13743 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
W20260811 05:10:56.375319 13743 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:43509 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:10:56.375445 13743 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.375605 13743 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:56.376608 13743 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.376770 13743 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
W20260811 05:10:56.377662 13743 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:10:56.476691 13747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:56.476943 13747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:56.477035 13747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:56.481768 13747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:56.481854 13747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:56.481946 13747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:10:56.488278 13747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:33723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=42111
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:56.489476 13747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:56.490597 13747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:56.501366 13753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:56.501394 13755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:56.501394 13752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:56.502441 13747 server_base.cc:1034] running on GCE node
I20260811 05:10:56.502833 13747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:56.503636 13747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:56.504808 13747 hybrid_clock.cc:648] HybridClock initialized: now 1786425056504755 us; error 71 us; skew 500 ppm
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:56.507660 13747 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:10:56.508723 13747 webserver.cc:460] Webserver started at http://127.30.88.194:42111/ using document root <none> and password file <none>
I20260811 05:10:56.509416 13747 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:56.509514 13747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:56.527201 13747 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:10:56.529842 13763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.531277 13747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:10:56.531424 13747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "a3f4c170105548cea5ef503328ba0ad8"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "bcc2710e6c438a8a6c5f76340c277acc"
server_key_iv: "63b0f38cf5b5cc54a1d0e87613fbc78d"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:56.532040 13747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:56.554095 13747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:56.557420 13747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:56.557693 13747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:56.558408 13747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:56.559407 13747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:56.559530 13747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.559633 13747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:56.559702 13747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.571370 13747 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:33723
I20260811 05:10:56.572429 13747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:10:56.573472 13887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:33723 every 8 connection(s)
I20260811 05:10:56.573514 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13747
I20260811 05:10:56.574026 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.88.212:44849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:33189
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:43509
--webserver_port=36053
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:56.594198 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.579217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:33615 (local address 127.30.88.254:43509)
0811 05:10:56.579352 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:56.579358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:56.579377 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:10:56.579944 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:56.579948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:56.580017 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:56.580133 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:56.581824 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.582748 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.584905 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.585311 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.585418 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.589118 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:56.589124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:56.589130 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:56.589165 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:56.591546 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.592296 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.592300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.592302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.592399 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.592802 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.592806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.592807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.593049 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:56.593134 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:56.593865 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 05:10:56.594005 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:10:56.595860 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.578086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.194:33615)
0811 05:10:56.578610 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:56.578666 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:56.579717 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:56.580477 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:56.580489 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:56.580893 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:56.581617 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.581641 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.583009 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.583016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.584712 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.584728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.585673 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.585679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.585967 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.586766 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:56.586794 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:56.588901 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:56.591720 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.591728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.591747 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.592157 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.592518 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.592520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.592522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.592688 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.593226 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:56.593233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:56.593546 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 05:10:56.594448 (+ 902us) client_negotiation.cc:240] Negotiation successful
0811 05:10:56.594780 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":164,"threads_started":1}
I20260811 05:10:56.597419 13888 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:56.597645 13888 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.598229 13888 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
W20260811 05:10:56.600030 13888 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:43509 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:10:56.600147 13888 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.600303 13888 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:56.601210 13888 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.601380 13888 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
W20260811 05:10:56.602207 13888 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:10:56.719973 13891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:10:56.720254 13891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:10:56.720319 13891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:10:56.725080 13891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:10:56.725200 13891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:10:56.725323 13891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:10:56.732074 13891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.88.212:44849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:33189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=36053
--tserver_master_addrs=127.30.88.254:43509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:10:56.733040 13891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:10:56.733958 13891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:10:56.743439 13902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:10:56.743800 13901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:56.744169 13891 server_base.cc:1034] running on GCE node
W20260811 05:10:56.744486 13904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:10:56.744957 13891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:10:56.745687 13891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:10:56.748989 13891 hybrid_clock.cc:648] HybridClock initialized: now 1786425056748956 us; error 44 us; skew 500 ppm
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:56.752532 13891 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:10:56.753585 13891 webserver.cc:460] Webserver started at http://127.30.88.195:36053/ using document root <none> and password file <none>
I20260811 05:10:56.754298 13891 fs_manager.cc:359] Metadata directory not provided
I20260811 05:10:56.754408 13891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:56.776172 13891 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260811 05:10:56.779260 13914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.780767 13891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:10:56.780920 13891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "b3630674701048ada1f4bdbe1b091584"
format_stamp: "Formatted at 2026-08-11 05:10:55 on dist-test-slave-7c35"
server_key: "00e1e918dfdfd6cb146c850496eba0f2"
server_key_iv: "60186cf1a09df9d35f7fd8dde8f93e93"
server_key_version: "kuduclusterkey@0"
I20260811 05:10:56.781522 13891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:10:56.796016 13891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:10:56.799293 13891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:10:56.799477 13891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:10:56.800061 13891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:10:56.800896 13891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:10:56.800953 13891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.801045 13891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:10:56.801090 13891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:10:56.812232 13891 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:33189
I20260811 05:10:56.813474 13891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:10:56.815049 14036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:33189 every 8 connection(s)
I20260811 05:10:56.816315 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 13891
Aug 11 05:10:56 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:10:56.829079 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.816677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:52283 (local address 127.30.88.254:43509)
0811 05:10:56.816819 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:56.816824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:56.817357 (+ 533us) server_negotiation.cc:407] Connection header received
0811 05:10:56.818275 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:56.818278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:56.818333 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:56.818420 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:56.819786 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.820316 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.821819 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.822083 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.822194 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.824701 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:56.824705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:56.824708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:56.824733 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:56.826453 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.827194 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.827198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.827200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.827301 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.827772 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.827775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.827777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.827999 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:56.828089 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:56.828784 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 05:10:56.828906 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:10:56.830916 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.815673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43509 (local address 127.30.88.195:52283)
0811 05:10:56.817211 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:56.817264 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:56.818076 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:56.818646 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:56.818655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:56.819026 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:10:56.819619 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.819637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.820442 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.820445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.821707 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:56.821718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.822245 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.822251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:56.822476 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.822998 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:10:56.823014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:10:56.824527 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:10:56.826607 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.826612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.826626 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.827064 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.827432 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:10:56.827435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:10:56.827437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:10:56.827637 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:10:56.828165 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:10:56.828171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:10:56.828468 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 05:10:56.829516 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0811 05:10:56.829866 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":140,"threads_started":1}
I20260811 05:10:56.833029 14037 heartbeater.cc:347] Connected to a master server at 127.30.88.254:43509
I20260811 05:10:56.833254 14037 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.833753 14037 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:56.835093 14041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:56.816903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.30.88.254:43509)
0811 05:10:56.817166 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:56.817171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:56.818919 (+ 1748us) server_negotiation.cc:407] Connection header received
0811 05:10:56.818963 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:56.818965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:56.819000 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:56.819094 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:56.820410 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.820857 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.821643 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:56.821926 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:56.822014 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:56.831483 (+ 9469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:56.831489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:56.831495 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:56.831524 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:56.833273 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.833869 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.833872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.833873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.833951 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:56.834236 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:56.834240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:56.834243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:56.834436 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:56.834620 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:56.834784 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 05:10:56.834934 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
W20260811 05:10:56.835355 14037 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:43509 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:10:56.835465 14037 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.835637 14037 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:56.836647 14037 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:56.836824 14037 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
W20260811 05:10:56.837595 14037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 05:10:57 dist-test-slave-7c35 krb5kdc[12418](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:10:57.377921 13743 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:57.378265 13743 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:57.379652 13520 ts_manager.cc:194] Registered new tserver with Master: 5f5f5212b2e043ca818d916a253660e0 (127.30.88.193:46865)
I20260811 05:10:57.381461 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:38225
Aug 11 05:10:57 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:57.602572 13888 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:57.602914 13888 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:57.603732 13520 ts_manager.cc:194] Registered new tserver with Master: a3f4c170105548cea5ef503328ba0ad8 (127.30.88.194:33723)
I20260811 05:10:57.604583 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:33615
Aug 11 05:10:57 dist-test-slave-7c35 krb5kdc[12418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for HTTP/127.30.88.212@KRBTEST.COM
I20260811 05:10:57.837828 14037 heartbeater.cc:464] Registering TS with master...
I20260811 05:10:57.838164 14037 heartbeater.cc:511] Master 127.30.88.254:43509 requested a full tablet report, sending...
I20260811 05:10:57.838924 13520 ts_manager.cc:194] Registered new tserver with Master: b3630674701048ada1f4bdbe1b091584 (127.30.88.195:33189)
I20260811 05:10:57.839704 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:52283
I20260811 05:10:57.849442 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:10:57.863919 14063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.853736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35162 (local address 127.30.88.254:43509)
0811 05:10:57.854566 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:57.854570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:57.854582 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:10:57.854618 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:57.854621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:57.854663 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:57.855568 (+ 905us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:10:57.855662 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.856423 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.857566 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.857929 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.858423 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.859991 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:10:57.859996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:10:57.859999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:10:57.860024 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:10:57.861760 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:57.862227 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:57.862230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:57.862232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:57.862309 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:10:57.862584 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:10:57.862587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:10:57.862588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:10:57.862776 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:10:57.863539 (+ 763us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:57.863609 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 05:10:57.863725 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":68,"threads_started":1}
I20260811 05:10:57.868153 13520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:10:57.872988 13520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:10:57.905581 14075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.888931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33723 (local address 127.0.0.1:54326)
0811 05:10:57.892424 (+ 3493us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:57.892471 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:57.892582 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:57.895156 (+ 2574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:57.895162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:57.895185 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:57.895314 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.895319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.899326 (+ 4007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.899332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.900863 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.900887 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.905066 (+ 4179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.905069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.905175 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.905220 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 05:10:57.905306 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 05:10:57.905422 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2661,"thread_start_us":110,"threads_started":1}
I20260811 05:10:57.907251 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.889094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.30.88.194:33723)
0811 05:10:57.894105 (+ 5011us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:57.894111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:57.894132 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:10:57.894213 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:57.894217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:57.894370 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:57.895680 (+ 1310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:57.895805 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.896955 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.901135 (+ 4180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.904143 (+ 3008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.906802 (+ 2659us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.906981 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:57.907033 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 05:10:57.907118 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4910,"thread_start_us":88,"threads_started":1}
I20260811 05:10:57.908483 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.891231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46865 (local address 127.0.0.1:53674)
0811 05:10:57.893165 (+ 1934us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:57.893177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:57.893245 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:57.895447 (+ 2202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:57.895449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:57.895464 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:57.895538 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.895543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.902053 (+ 6510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.902057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.903183 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.903194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.908127 (+ 4933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.908130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.908220 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.908244 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:10:57.908288 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 05:10:57.908348 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1854,"thread_start_us":99,"threads_started":1}
I20260811 05:10:57.909319 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.891419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.30.88.193:46865)
0811 05:10:57.893536 (+ 2117us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:57.893543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:57.893565 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:10:57.893639 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:57.893644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:57.893831 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:57.897111 (+ 3280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:57.897234 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.901000 (+ 3766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.903328 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.904923 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.908914 (+ 3991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.909061 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:57.909107 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 05:10:57.909187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2017,"thread_start_us":92,"threads_started":1}
I20260811 05:10:57.912798 13816 tablet_service.cc:1467] Processing CreateTablet for tablet c027769320ad4261adc88407cdefcd8c (DEFAULT_TABLE table=test-table [id=c13bcad32acb4d3f9f18d0678d2045cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:57.912895 13669 tablet_service.cc:1467] Processing CreateTablet for tablet c027769320ad4261adc88407cdefcd8c (DEFAULT_TABLE table=test-table [id=c13bcad32acb4d3f9f18d0678d2045cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:57.913944 13669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c027769320ad4261adc88407cdefcd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:57.913946 13816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c027769320ad4261adc88407cdefcd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:57.914458 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.890604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:33189 (local address 127.0.0.1:36852)
0811 05:10:57.892812 (+ 2208us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:57.892830 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:57.892953 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:57.909735 (+ 16782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:57.909738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:57.909757 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:57.909838 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.909844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.912068 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.912071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.913058 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:57.913069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.914114 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.914118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:57.914201 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.914223 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:10:57.914268 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:10:57.914325 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2141,"thread_start_us":93,"threads_started":1}
I20260811 05:10:57.915117 14074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:57.890765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.30.88.195:33189)
0811 05:10:57.907636 (+ 16871us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:57.907643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:57.907667 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:10:57.907740 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:57.907745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:57.907957 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:57.910002 (+ 2045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:57.910193 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.911266 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.913226 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:57.913959 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:57.914692 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:57.914820 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:57.914903 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 05:10:57.914985 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16764,"thread_start_us":160,"threads_started":1}
I20260811 05:10:57.917687 13965 tablet_service.cc:1467] Processing CreateTablet for tablet c027769320ad4261adc88407cdefcd8c (DEFAULT_TABLE table=test-table [id=c13bcad32acb4d3f9f18d0678d2045cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:10:57.918689 13965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c027769320ad4261adc88407cdefcd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:10:57.934669 14079 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Bootstrap starting.
I20260811 05:10:57.935117 14078 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Bootstrap starting.
I20260811 05:10:57.938781 14079 tablet_bootstrap.cc:654] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:57.940018 14078 tablet_bootstrap.cc:654] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:57.940969 14079 log.cc:824] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:57.941828 14078 log.cc:824] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:57.943794 14077 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Bootstrap starting.
I20260811 05:10:57.945075 14079 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: No bootstrap required, opened a new log
I20260811 05:10:57.945276 14079 ts_tablet_manager.cc:1397] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 05:10:57.947742 14077 tablet_bootstrap.cc:654] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Neither blocks nor log segments found. Creating new log.
I20260811 05:10:57.948544 14077 log.cc:824] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:10:57.949399 14079 raft_consensus.cc:348] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.949673 14079 raft_consensus.cc:374] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:57.949719 14079 raft_consensus.cc:729] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f4c170105548cea5ef503328ba0ad8, State: Initialized, Role: FOLLOWER
I20260811 05:10:57.950455 14079 consensus_queue.cc:260] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.953105 14078 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: No bootstrap required, opened a new log
I20260811 05:10:57.953305 14078 ts_tablet_manager.cc:1397] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260811 05:10:57.957017 14077 tablet_bootstrap.cc:492] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: No bootstrap required, opened a new log
I20260811 05:10:57.957327 14077 ts_tablet_manager.cc:1397] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260811 05:10:57.957448 14078 raft_consensus.cc:348] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.957775 14078 raft_consensus.cc:374] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:57.957916 14078 raft_consensus.cc:729] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3630674701048ada1f4bdbe1b091584, State: Initialized, Role: FOLLOWER
I20260811 05:10:57.958485 14078 consensus_queue.cc:260] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.959609 14078 ts_tablet_manager.cc:1428] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:10:57.960280 14079 ts_tablet_manager.cc:1428] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260811 05:10:57.960548 14037 heartbeater.cc:503] Master 127.30.88.254:43509 was elected leader, sending a full tablet report...
I20260811 05:10:57.964612 13888 heartbeater.cc:503] Master 127.30.88.254:43509 was elected leader, sending a full tablet report...
I20260811 05:10:57.969072 14077 raft_consensus.cc:348] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.969352 14077 raft_consensus.cc:374] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:10:57.969415 14077 raft_consensus.cc:729] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f5f5212b2e043ca818d916a253660e0, State: Initialized, Role: FOLLOWER
I20260811 05:10:57.969851 14077 consensus_queue.cc:260] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:57.970497 13743 heartbeater.cc:503] Master 127.30.88.254:43509 was elected leader, sending a full tablet report...
I20260811 05:10:57.970984 14077 ts_tablet_manager.cc:1428] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260811 05:10:58.044850 14085 raft_consensus.cc:482] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:10:58.045102 14085 raft_consensus.cc:504] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:58.046586 14085 leader_election.cc:288] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3f4c170105548cea5ef503328ba0ad8 (127.30.88.194:33723), 5f5f5212b2e043ca818d916a253660e0 (127.30.88.193:46865)
I20260811 05:10:58.056680 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.047165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:48299 (local address 127.30.88.194:33723)
0811 05:10:58.047284 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:58.047288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:58.048123 (+ 835us) server_negotiation.cc:407] Connection header received
0811 05:10:58.048331 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:58.048335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:58.048381 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:58.048430 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:58.048920 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.049704 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.054809 (+ 5105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.055526 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.055622 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:58.055675 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:58.056455 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 05:10:58.056543 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:10:58.057331 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.046763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33723 (local address 127.30.88.195:48299)
0811 05:10:58.048035 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:58.048050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:58.048205 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:58.048576 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:58.048579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:58.048597 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:58.048762 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:58.048770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.049857 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.049860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:58.054659 (+ 4799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:58.054672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.055775 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.055779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:58.055882 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:58.055920 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 05:10:58.057098 (+ 1178us) client_negotiation.cc:240] Negotiation successful
0811 05:10:58.057183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"thread_start_us":139,"threads_started":1}
I20260811 05:10:58.058185 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c027769320ad4261adc88407cdefcd8c" candidate_uuid: "b3630674701048ada1f4bdbe1b091584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f4c170105548cea5ef503328ba0ad8" is_pre_election: true
I20260811 05:10:58.058548 13836 raft_consensus.cc:2454] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3630674701048ada1f4bdbe1b091584 in term 0.
I20260811 05:10:58.059509 13915 leader_election.cc:302] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3f4c170105548cea5ef503328ba0ad8, b3630674701048ada1f4bdbe1b091584; no voters:
I20260811 05:10:58.059800 14085 raft_consensus.cc:2781] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:10:58.059867 14085 raft_consensus.cc:482] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:10:58.059916 14085 raft_consensus.cc:3037] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:58.061318 14085 raft_consensus.cc:504] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
I20260811 05:10:58.061662 14085 leader_election.cc:288] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [CANDIDATE]: Term 1 election: Requested vote from peers a3f4c170105548cea5ef503328ba0ad8 (127.30.88.194:33723), 5f5f5212b2e043ca818d916a253660e0 (127.30.88.193:46865)
I20260811 05:10:58.062297 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c027769320ad4261adc88407cdefcd8c" candidate_uuid: "b3630674701048ada1f4bdbe1b091584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f4c170105548cea5ef503328ba0ad8"
I20260811 05:10:58.062412 13836 raft_consensus.cc:3037] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:10:58.063606 13836 raft_consensus.cc:2454] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3630674701048ada1f4bdbe1b091584 in term 1.
I20260811 05:10:58.063963 13915 leader_election.cc:302] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3f4c170105548cea5ef503328ba0ad8, b3630674701048ada1f4bdbe1b091584; no voters:
I20260811 05:10:58.064241 14085 raft_consensus.cc:2781] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:10:58.064433 14085 raft_consensus.cc:686] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 1 LEADER]: Becoming Leader. State: Replica: b3630674701048ada1f4bdbe1b091584, State: Running, Role: LEADER
I20260811 05:10:58.064775 14085 consensus_queue.cc:237] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } }
W20260811 05:10:58.065233 14038 tablet.cc:2369] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:10:58.068320 13518 catalog_manager.cc:5495] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3630674701048ada1f4bdbe1b091584 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "b3630674701048ada1f4bdbe1b091584" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3630674701048ada1f4bdbe1b091584" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 } health_report { overall_health: UNKNOWN } } }
I20260811 05:10:58.069453 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.047690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:54283 (local address 127.30.88.193:46865)
0811 05:10:58.047879 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:58.047884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:58.056218 (+ 8334us) server_negotiation.cc:407] Connection header received
0811 05:10:58.056365 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:58.056368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:58.056426 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:58.056508 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:10:58.058313 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.059063 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.068100 (+ 9037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.068718 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.068843 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:58.068905 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:58.069247 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 05:10:58.069323 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 05:10:58.070103 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.047559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46865 (local address 127.30.88.195:54283)
0811 05:10:58.056132 (+ 8573us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:58.056146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:10:58.056251 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:10:58.057974 (+ 1723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:10:58.057977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:10:58.057998 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:10:58.058104 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:58.058110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.066811 (+ 8701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.066814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:58.067955 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:10:58.067967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.068971 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.068974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:10:58.069094 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:58.069117 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:10:58.069892 (+ 775us) client_negotiation.cc:240] Negotiation successful
0811 05:10:58.069962 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8494,"thread_start_us":93,"threads_started":1}
I20260811 05:10:58.070950 13676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c027769320ad4261adc88407cdefcd8c" candidate_uuid: "b3630674701048ada1f4bdbe1b091584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5f5212b2e043ca818d916a253660e0" is_pre_election: true
I20260811 05:10:58.071031 13690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c027769320ad4261adc88407cdefcd8c" candidate_uuid: "b3630674701048ada1f4bdbe1b091584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5f5212b2e043ca818d916a253660e0"
I20260811 05:10:58.071218 13676 raft_consensus.cc:2454] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3630674701048ada1f4bdbe1b091584 in term 0.
W20260811 05:10:58.075047 13890 tablet.cc:2369] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:10:58.083751 14074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.079733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.30.88.195:33189)
0811 05:10:58.079868 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:10:58.079872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:10:58.080056 (+ 184us) server_negotiation.cc:407] Connection header received
0811 05:10:58.080261 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:10:58.080265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:10:58.080332 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:10:58.080451 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:10:58.080760 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.081252 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.082124 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:10:58.082364 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:10:58.082483 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:10:58.082814 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:10:58.083290 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:10:58.083452 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 05:10:58.083522 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 05:10:58.083579 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:10:58.089864 13836 raft_consensus.cc:1261] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 1 FOLLOWER]: Refusing update from remote peer b3630674701048ada1f4bdbe1b091584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:10:58.090598 14085 consensus_queue.cc:1035] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f4c170105548cea5ef503328ba0ad8" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:10:58.090651 13690 raft_consensus.cc:3037] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 0 FOLLOWER]: Advancing to term 1
W20260811 05:10:58.090957 13744 tablet.cc:2369] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:10:58.092278 13690 raft_consensus.cc:1261] T c027769320ad4261adc88407cdefcd8c P 5f5f5212b2e043ca818d916a253660e0 [term 1 FOLLOWER]: Refusing update from remote peer b3630674701048ada1f4bdbe1b091584: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:10:58.092872 14085 consensus_queue.cc:1035] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f5f5212b2e043ca818d916a253660e0" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 46865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:10:58.098446 14099 mvcc.cc:205] Tried to move back new op lower bound from 7317197037928304640 to 7317197037837234176. Current Snapshot: MvccSnapshot[applied={T|T < 7317197037928304640}]
I20260811 05:10:58.106905 13518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:10:58.107054 13518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:10:58.109536 13518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04832409d53d4634b7497024c2affd2d: Sending DeleteTablet for 3 replicas of tablet c027769320ad4261adc88407cdefcd8c
I20260811 05:10:58.110414 13965 tablet_service.cc:1514] Processing DeleteTablet for tablet c027769320ad4261adc88407cdefcd8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:58 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:36852
I20260811 05:10:58.110430 13816 tablet_service.cc:1514] Processing DeleteTablet for tablet c027769320ad4261adc88407cdefcd8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:58 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:54326
I20260811 05:10:58.110980 14109 tablet_replica.cc:330] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: stopping tablet replica
I20260811 05:10:58.111021 13669 tablet_service.cc:1514] Processing DeleteTablet for tablet c027769320ad4261adc88407cdefcd8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:10:58 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:53674
I20260811 05:10:58.111349 14109 raft_consensus.cc:2229] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:10:58.111671 14109 raft_consensus.cc:2258] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:10:58.112064 14110 tablet_replica.cc:330] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: stopping tablet replica
I20260811 05:10:58.112205 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13565
I20260811 05:10:58.125825 14110 raft_consensus.cc:2229] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:10:58.126163 14110 raft_consensus.cc:2258] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:10:58.128203 14110 ts_tablet_manager.cc:1905] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:10:58.130865 14110 ts_tablet_manager.cc:1918] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:10:58.130962 14110 log.cc:1196] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/c027769320ad4261adc88407cdefcd8c
I20260811 05:10:58.131289 14110 ts_tablet_manager.cc:1939] T c027769320ad4261adc88407cdefcd8c P a3f4c170105548cea5ef503328ba0ad8: Deleting consensus metadata
I20260811 05:10:58.132921 13502 catalog_manager.cc:4841] TS a3f4c170105548cea5ef503328ba0ad8 (127.30.88.194:33723): tablet c027769320ad4261adc88407cdefcd8c (table test-table [id=c13bcad32acb4d3f9f18d0678d2045cc]) successfully deleted
I20260811 05:10:58.135561 14109 ts_tablet_manager.cc:1905] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:10:58.137519 13505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:46865 (error 108)
I20260811 05:10:58.138415 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13747
I20260811 05:10:58.138428 14109 ts_tablet_manager.cc:1918] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:10:58.146283 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:10:58.145941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:46865 (local address 127.0.0.1:53690)
0811 05:10:58.146125 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 05:10:58.146194 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260811 05:10:58.146533 13505 catalog_manager.cc:4584] TS 5f5f5212b2e043ca818d916a253660e0 (127.30.88.193:46865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c027769320ad4261adc88407cdefcd8c: Network error: Client connection negotiation failed: client connection to 127.30.88.193:46865: connect: Connection refused (error 111)
I20260811 05:10:58.152110 14109 log.cc:1196] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/c027769320ad4261adc88407cdefcd8c
I20260811 05:10:58.152562 14109 ts_tablet_manager.cc:1939] T c027769320ad4261adc88407cdefcd8c P b3630674701048ada1f4bdbe1b091584: Deleting consensus metadata
I20260811 05:10:58.153982 13505 catalog_manager.cc:4841] TS b3630674701048ada1f4bdbe1b091584 (127.30.88.195:33189): tablet c027769320ad4261adc88407cdefcd8c (table test-table [id=c13bcad32acb4d3f9f18d0678d2045cc]) successfully deleted
I20260811 05:10:58.158640 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13891
I20260811 05:10:58.167459 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 13484
I20260811 05:10:58.177057 31075 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:10:58.522626 31075 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:10:58.522739 31075 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:10:58.868448 31075 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:10:58.868 UTC [12453] LOG: received smart shutdown request
2026-08-11 05:10:58.872 UTC [12453] LOG: background worker "logical replication launcher" (PID 12462) exited with exit code 1
2026-08-11 05:10:58.873 UTC [12457] LOG: shutting down
2026-08-11 05:10:59.032 UTC [12453] LOG: database system is shut down
2026-08-11T05:10:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47147 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:10:59 dist-test-slave-7c35 krb5kdc[14123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:10:59 dist-test-slave-7c35 krb5kdc[14123](info): set up 2 sockets
Aug 11 05:10:59 dist-test-slave-7c35 krb5kdc[14123](info): commencing operation
krb5kdc: starting...
W20260811 05:11:00.227085 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:11:00.341966 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43589
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:00.448698 14139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:00.448966 14139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:00.449046 14139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:00.453013 14139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:00.453094 14139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:00.453133 14139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:00.453191 14139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:00.453231 14139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:00.458120 14139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:00.459087 14139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:00.460011 14139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:00.468080 14144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:00.468140 14145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:00.468343 14139 server_base.cc:1034] running on GCE node
W20260811 05:11:00.468197 14147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:00.468906 14139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:00.469817 14139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:00.470999 14139 hybrid_clock.cc:648] HybridClock initialized: now 1786425060470975 us; error 32 us; skew 500 ppm
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:00.473716 14139 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:00.474462 14139 webserver.cc:460] Webserver started at http://127.30.88.254:37313/ using document root <none> and password file <none>
I20260811 05:11:00.475076 14139 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:00.475153 14139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:00.475361 14139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:00.477164 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "7e4f82eb4ada429c92d65eac3df85ad6"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "47cb3b6cc6d167761743148bce28554a"
server_key_iv: "ee929ba03ea1a3f40a4c5b9767e2284f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.477698 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "7e4f82eb4ada429c92d65eac3df85ad6"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "47cb3b6cc6d167761743148bce28554a"
server_key_iv: "ee929ba03ea1a3f40a4c5b9767e2284f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.481222 14139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:11:00.483420 14154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:00.484537 14139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:11:00.484685 14139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "7e4f82eb4ada429c92d65eac3df85ad6"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "47cb3b6cc6d167761743148bce28554a"
server_key_iv: "ee929ba03ea1a3f40a4c5b9767e2284f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.484797 14139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:00.500137 14139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:00.503408 14139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:00.503613 14139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:00.510233 14139 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:43589
I20260811 05:11:00.510262 14212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:43589 every 8 connection(s)
I20260811 05:11:00.511144 14139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:00.513648 14213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:00.517207 14213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:11:00.517587 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14139
I20260811 05:11:00.517709 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:11:00.518002 31075 external_mini_cluster.cc:1428] Setting key 6de11146ecfb4d5c3d693ea1e4027f60
WARNING: no policy specified for kudu/127.30.88.253@KRBTEST.COM; defaulting to no policy
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.253@KRBTEST.COM" created.
I20260811 05:11:00.532975 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.519757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43589 (local address 127.0.0.1:33698)
0811 05:11:00.520219 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.520269 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.521055 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.521847 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.521851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.522160 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.522406 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.522419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.524262 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.524264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.524915 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.524923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.525835 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.525837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.525929 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.526487 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.526502 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.528064 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.529876 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.529884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.529891 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.530194 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.530596 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.530598 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.530599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.530752 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.531391 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.531400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.531583 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.531772 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.532303 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":189,"thread_start_us":135,"threads_started":1}
I20260811 05:11:00.532980 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.519757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.30.88.254:43589)
0811 05:11:00.520319 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.520334 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.520400 (+ 66us) server_negotiation.cc:407] Connection header received
0811 05:11:00.521400 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.521408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.521644 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.521840 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.522563 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.524130 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.525036 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.525686 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.525849 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.528242 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.528247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.528256 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.528278 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.529781 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.530313 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.530321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.530325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.530494 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.530889 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.530893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.530895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.531267 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.531368 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.531844 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.532232 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":435,"spinlock_wait_cycles":1280,"thread_start_us":134,"threads_started":1}
I20260811 05:11:00.535109 14213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:11:00.535763 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.535540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48128)
0811 05:11:00.535643 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.535715 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:11:00.535929 14155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
I20260811 05:11:00.536649 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.536458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48134)
0811 05:11:00.536555 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.536596 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:11:00.536798 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:36375: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
I20260811 05:11:00.545217 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:36375
--webserver_interface=127.30.88.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:00.586030 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } attempt: 1
I20260811 05:11:00.587046 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.586648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48148)
0811 05:11:00.586852 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.586954 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 05:11:00.587787 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.587556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48162)
0811 05:11:00.587667 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.587721 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:11:00.587960 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:36375: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
W20260811 05:11:00.650353 14221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:00.650578 14221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:00.650616 14221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:00.654361 14221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:00.654428 14221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:00.654446 14221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:00.654461 14221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:00.654475 14221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:00.656126 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } attempt: 2
I20260811 05:11:00.656971 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.656661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48172)
0811 05:11:00.656821 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.656885 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 05:11:00.657636 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.657441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48176)
0811 05:11:00.657536 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.657580 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 05:11:00.657773 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:36375: Network error: Client connection negotiation failed: client connection to 127.30.88.253:36375: connect: Connection refused (error 111)
I20260811 05:11:00.659890 14221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.88.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:00.660802 14221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:00.661703 14221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:00.669947 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:00.669938 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:00.669967 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:00.670091 14221 server_base.cc:1034] running on GCE node
I20260811 05:11:00.670742 14221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:00.671818 14221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:00.673015 14221 hybrid_clock.cc:648] HybridClock initialized: now 1786425060673007 us; error 27 us; skew 500 ppm
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:00.675702 14221 init.cc:377] Logged in from keytab as kudu/127.30.88.253@KRBTEST.COM (short username kudu)
I20260811 05:11:00.676510 14221 webserver.cc:460] Webserver started at http://127.30.88.253:41921/ using document root <none> and password file <none>
I20260811 05:11:00.677009 14221 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:00.677062 14221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:00.677209 14221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:00.679114 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data/instance:
uuid: "5ef07ba69640423c9c7dee3de6cb6d4c"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "de3d55a64ecc71f635053718068aede2"
server_key_iv: "e3bda04c605659d4c5d12483193ec62f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.679636 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal/instance:
uuid: "5ef07ba69640423c9c7dee3de6cb6d4c"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "de3d55a64ecc71f635053718068aede2"
server_key_iv: "e3bda04c605659d4c5d12483193ec62f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.683163 14221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:11:00.685485 14239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:00.686617 14221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:00.686762 14221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "5ef07ba69640423c9c7dee3de6cb6d4c"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "de3d55a64ecc71f635053718068aede2"
server_key_iv: "e3bda04c605659d4c5d12483193ec62f"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.686918 14221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:00.696439 14221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:00.699059 14221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:00.699244 14221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:00.705967 14221 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.253:36375
I20260811 05:11:00.705978 14298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.253:36375 every 8 connection(s)
I20260811 05:11:00.706774 14221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
I20260811 05:11:00.709384 14299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:00.710696 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14221
I20260811 05:11:00.710824 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-1/wal/instance
I20260811 05:11:00.711126 31075 external_mini_cluster.cc:1428] Setting key f4177f8c64e65bdc1f2f1d322ca0c7c8
I20260811 05:11:00.713925 14299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.88.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.252@KRBTEST.COM" created.
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:00.726905 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.715624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.30.88.254:43589)
0811 05:11:00.715776 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.715780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.716725 (+ 945us) server_negotiation.cc:407] Connection header received
0811 05:11:00.717594 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.717598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.717666 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.717771 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.718836 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.719425 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.720693 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.720989 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.721126 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.723596 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.723600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.723602 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.723627 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.724976 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.725471 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.725475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.725476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.725556 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.725868 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.725872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.725873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.726040 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.726150 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.726597 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.726713 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:11:00.727576 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.716067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43589 (local address 127.0.0.1:33706)
0811 05:11:00.716547 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.716624 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.717385 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.717926 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.717931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.718251 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.718686 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.718700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.719557 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.719559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.720501 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.720511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.721162 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.721166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.721341 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.721927 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.721942 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.723436 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.725124 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.725129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.725141 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.725368 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.725645 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.725648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.725649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.725776 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.726150 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.726156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.726374 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.726540 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.726814 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260811 05:11:00.729090 14299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
I20260811 05:11:00.737581 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.729517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48184)
0811 05:11:00.729616 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.729631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.729699 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.730335 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.730338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.730502 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.730625 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.730630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.731668 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.731673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.732123 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.732135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.732640 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.732643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.732718 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.733089 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.733103 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.734229 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.736066 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.736070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.736074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.736302 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.736621 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.736625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.736629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.736715 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.737215 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.737217 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.737299 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.737395 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.737483 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 05:11:00.737669 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:33513
--webserver_interface=127.30.88.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:00.737847 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.729671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.30.88.253:36375)
0811 05:11:00.729981 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.729994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.730020 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:11:00.730052 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.730055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.730215 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.730353 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.730744 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.731565 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.732233 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.732539 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.732641 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.734348 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.734357 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.734375 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.734393 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.735933 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.736407 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.736412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.736414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.736490 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.736808 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.736811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.736812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.737092 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.737228 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.737505 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.737647 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":161,"threads_started":1}
I20260811 05:11:00.739744 14299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:11:00.740494 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.740185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:40984)
0811 05:11:00.740318 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.740397 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:11:00.740696 14240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
I20260811 05:11:00.741537 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.741266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:40986)
0811 05:11:00.741406 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.741465 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 05:11:00.741694 14299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
I20260811 05:11:00.748939 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } attempt: 3
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:11:00.757220 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.749317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48190)
0811 05:11:00.749456 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.749471 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.749552 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.749785 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.749788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.749990 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.750123 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.750130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.750786 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.750790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.752145 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.752157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.752574 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.752577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.752682 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.753087 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.753103 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.754256 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.755873 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.755877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.755879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.756059 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.756320 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.756323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.756325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.756412 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.756765 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.756768 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.756860 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.756968 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.757081 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 05:11:00.757220 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.749413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.30.88.253:36375)
0811 05:11:00.749507 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.749510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.749522 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:00.749650 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.749652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.749685 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.749746 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.750232 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.750690 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.752257 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.752471 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.752546 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.754385 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.754388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.754390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.754409 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.755763 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.756153 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.756157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.756159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.756231 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.756493 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.756497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.756499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.756664 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.756770 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.757013 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.757112 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 05:11:00.758152 14213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:11:00.758782 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.758564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:40998)
0811 05:11:00.758674 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.758721 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:11:00.759472 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.759224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41006)
0811 05:11:00.759367 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.759407 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 05:11:00.759606 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
I20260811 05:11:00.790750 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 1
I20260811 05:11:00.790961 14299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 1
I20260811 05:11:00.791678 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.791313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41008)
0811 05:11:00.791499 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.791589 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 05:11:00.791709 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.791410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41016)
0811 05:11:00.791592 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.791640 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 05:11:00.792395 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.792147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41022)
0811 05:11:00.792271 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.792326 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:11:00.792531 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
I20260811 05:11:00.792546 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.792317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41036)
0811 05:11:00.792426 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.792482 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 05:11:00.792658 14299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
W20260811 05:11:00.845971 14305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:00.846174 14305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:00.846208 14305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:00.849807 14305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:00.849848 14305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:00.849864 14305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:00.849879 14305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:00.849890 14305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:00.854409 14305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43589,127.30.88.253:36375,127.30.88.252:33513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:33513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.88.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:00.855261 14305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:00.856117 14305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:11:00.860839 14299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 2
I20260811 05:11:00.861619 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.861345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41042)
0811 05:11:00.861482 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.861543 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 05:11:00.862334 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.862109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41058)
0811 05:11:00.862221 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.862267 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:11:00.862581 14299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
W20260811 05:11:00.863497 14311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:00.863497 14314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:00.863528 14312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:00.863899 14305 server_base.cc:1034] running on GCE node
I20260811 05:11:00.864449 14305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:00.865418 14305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:00.866582 14305 hybrid_clock.cc:648] HybridClock initialized: now 1786425060866555 us; error 36 us; skew 500 ppm
I20260811 05:11:00.867672 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 2
I20260811 05:11:00.868345 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.868081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41060)
0811 05:11:00.868224 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.868274 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 05:11:00.869019 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.868803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41066)
0811 05:11:00.868921 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.868968 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:11:00.869235 14213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:33513: Network error: Client connection negotiation failed: client connection to 127.30.88.252:33513: connect: Connection refused (error 111)
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:00.869755 14305 init.cc:377] Logged in from keytab as kudu/127.30.88.252@KRBTEST.COM (short username kudu)
I20260811 05:11:00.870525 14305 webserver.cc:460] Webserver started at http://127.30.88.252:38273/ using document root <none> and password file <none>
I20260811 05:11:00.871060 14305 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:00.871121 14305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:00.871273 14305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:00.873188 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data/instance:
uuid: "ac6537e8203d4c9a8ecf033a5c174017"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "2e418066d0a472fec87c6b34a5a6d42c"
server_key_iv: "a1b8c08f8a901ecd915336bd21cb24ac"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.873740 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal/instance:
uuid: "ac6537e8203d4c9a8ecf033a5c174017"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "2e418066d0a472fec87c6b34a5a6d42c"
server_key_iv: "a1b8c08f8a901ecd915336bd21cb24ac"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.877202 14305 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:11:00.879558 14321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:00.880716 14305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:00.880848 14305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "ac6537e8203d4c9a8ecf033a5c174017"
format_stamp: "Formatted at 2026-08-11 05:11:00 on dist-test-slave-7c35"
server_key: "2e418066d0a472fec87c6b34a5a6d42c"
server_key_iv: "a1b8c08f8a901ecd915336bd21cb24ac"
server_key_version: "encryptionkey@0"
I20260811 05:11:00.880967 14305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:00.906761 14305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:00.909379 14305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:00.909577 14305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:00.916822 14305 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.252:33513
I20260811 05:11:00.916865 14378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.252:33513 every 8 connection(s)
I20260811 05:11:00.917723 14305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
I20260811 05:11:00.920644 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:00.924100 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14305
I20260811 05:11:00.924242 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786424902138073-31075-0/minicluster-data/master-2/wal/instance
I20260811 05:11:00.924547 31075 external_mini_cluster.cc:1428] Setting key 046baa4cfa8e58d4e256411e8f8cfe06
I20260811 05:11:00.924854 14379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:11:00.941229 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.926928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.30.88.254:43589)
0811 05:11:00.927077 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.927083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.927350 (+ 267us) server_negotiation.cc:407] Connection header received
0811 05:11:00.927667 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.927671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.927732 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.927843 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.928759 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.929336 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.930436 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.930723 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.930817 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.937370 (+ 6553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.937373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.937375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.937396 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.938968 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.939568 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.939570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.939571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.939639 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.939981 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.939986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.939989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.940247 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.940360 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.940902 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.941020 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:11:00.942075 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.925911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.30.88.252:33513)
0811 05:11:00.926341 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.926353 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.926396 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:11:00.927144 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.927166 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.927479 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.927697 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.928633 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.929588 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.930535 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.930985 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.931084 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.937727 (+ 6643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.937736 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.937749 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.937776 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.939727 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.940171 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.940175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.940176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.940241 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.940537 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.940540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.940541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.940798 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.940962 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.941229 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.941352 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":302,"thread_start_us":143,"threads_started":1}
I20260811 05:11:00.942080 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.926809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43589 (local address 127.0.0.1:33712)
0811 05:11:00.927203 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.927245 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.927466 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.927908 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.927914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.928335 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.928596 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.928618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.929482 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.929485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.930292 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.930302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.930895 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.930901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.931049 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.935721 (+ 4672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.935735 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.937212 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.939090 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.939097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.939107 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.939475 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.939723 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.939726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.939728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.939886 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.940371 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.940376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.940627 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.940817 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.941143 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":157,"threads_started":1}
I20260811 05:11:00.943544 14379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:00.951745 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.944029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:36375 (local address 127.0.0.1:48206)
0811 05:11:00.944145 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.944159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.944239 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.944736 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.944741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.944911 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.945029 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.945038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.945835 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.945837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.946446 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.946455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.946823 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.946827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.946946 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.947336 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.947346 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.948402 (+ 1056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.950284 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.950288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.950289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.950540 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.950796 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.950800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.950801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.950925 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.951266 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.951269 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.951381 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.951487 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.951586 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:11:00.951821 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.944290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.30.88.253:36375)
0811 05:11:00.944394 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.944397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.944407 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:11:00.944468 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.944471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.944518 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.944603 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.945150 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.945734 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.946552 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.946724 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.946793 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.948548 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.948551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.948553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.948582 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.950113 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.950632 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.950635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.950636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.950703 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.951015 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.951017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.951018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.951168 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.951243 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.951557 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.951669 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:11:00.952416 14379 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:11:00.952864 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.944577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.30.88.254:43589)
0811 05:11:00.944678 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.944680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.944691 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:11:00.944773 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.944775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.944803 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.944857 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.945484 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.946022 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.946900 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.947102 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.947172 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.949372 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.949388 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.949390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.949411 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.951011 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.951482 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.951489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.951491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.951581 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.952024 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.952027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.952028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.952249 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.952497 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.952630 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.952728 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:11:00.953743 14299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 3
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:11:00.961720 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.952853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41092)
0811 05:11:00.952946 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.952958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.953018 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.953378 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.953381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.953548 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.953652 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.953658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.954805 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.954810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.955535 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.955544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.955963 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.955966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.956032 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.957217 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.957227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.958323 (+ 1096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.960207 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.960209 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.960210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.960412 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.960691 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.960694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.960695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.960851 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.961260 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.961265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.961345 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.961459 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.961585 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":2}
I20260811 05:11:00.962201 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.953081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.30.88.252:33513)
0811 05:11:00.953149 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.953189 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.953201 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:00.953228 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.953230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.953269 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.953342 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.953757 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.954671 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.955639 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.955866 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.955965 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.958460 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.958463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.958466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.958486 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.960111 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.960507 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.960512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.960513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.960613 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.960982 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.960985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.960986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.961143 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.961249 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.961927 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.962044 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:11:00.964123 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.954105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41102)
0811 05:11:00.954194 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.954207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.954296 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.954898 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.954901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.955070 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.955132 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.955137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.956387 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.956390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.957099 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.957108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.957858 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.957861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.957955 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.958445 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.958466 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.960521 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.962462 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.962468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.962471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.962745 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.963104 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.963107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.963108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.963202 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.963639 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.963642 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.963732 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.963855 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.963977 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 05:11:00.964193 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.954385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.30.88.252:33513)
0811 05:11:00.954681 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.954691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.954703 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:00.954742 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.954745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.954781 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.954890 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.955235 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.956309 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.957194 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.957503 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.957633 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.960643 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.960649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.960652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.960676 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.962348 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.962890 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.962896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.962898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.962985 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.963280 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.963283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.963285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.963525 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.963642 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.963906 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.964007 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":229,"thread_start_us":124,"threads_started":1}
I20260811 05:11:00.964826 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.954077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.30.88.253:36375)
0811 05:11:00.954178 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.954182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.954194 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:00.954579 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.954584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.954628 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.954708 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.955388 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.956089 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.957021 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.957264 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.957351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.960706 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.960709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.960711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.960773 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.962633 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.963234 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.963237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.963239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.963327 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.963626 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.963628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.963630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.963824 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.964022 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.964593 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.964695 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:11:00.965732 14379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: Bootstrap starting.
I20260811 05:11:00.966917 14299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: Bootstrap starting.
I20260811 05:11:00.968128 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:11:00.968405 14213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } attempt: 3
I20260811 05:11:00.968590 14379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:00.969607 14299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:00.969676 14379 log.cc:824] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:00.970397 14299 log.cc:824] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:00.973615 14299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: No bootstrap required, opened a new log
I20260811 05:11:00.973654 14379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: No bootstrap required, opened a new log
I20260811 05:11:00.977505 14379 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:00.977769 14379 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:00.978497 14299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:00.978724 14299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:00.979319 14299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef07ba69640423c9c7dee3de6cb6d4c, State: Initialized, Role: FOLLOWER
I20260811 05:11:00.979908 14299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
Aug 11 05:11:00 dist-test-slave-7c35 krb5kdc[14123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:11:00.980489 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:00.980690 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:00.981405 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.971140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.30.88.254:43589)
0811 05:11:00.972030 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.972040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.972053 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:11:00.972100 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.972103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.972141 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.972214 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.973122 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.973714 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.974719 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.975050 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.975159 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.976923 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.976927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.976930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.976958 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.978574 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.979091 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.979095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.979096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.979682 (+ 586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.980019 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.980023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.980024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.980233 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.980328 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.981010 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.981277 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801}
I20260811 05:11:00.981873 14299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:00.983551 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.971354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.30.88.253:36375)
0811 05:11:00.971441 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.971445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.974893 (+ 3448us) server_negotiation.cc:407] Connection header received
0811 05:11:00.975309 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.975312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.975349 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.975392 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.976315 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.977008 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.978147 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.978463 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.978544 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.980126 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.980129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.980132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.980153 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.981679 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.982214 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.982219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.982220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.982302 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.982596 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.982599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.982601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.982778 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.982911 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.983291 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.983424 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:11:00.984496 14379 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6537e8203d4c9a8ecf033a5c174017, State: Initialized, Role: FOLLOWER
I20260811 05:11:00.986073 14379 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:00.987154 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.968754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:33513 (local address 127.0.0.1:41106)
0811 05:11:00.968866 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:00.968882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:00.968959 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:00.970347 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:00.970350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:00.970497 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:00.970616 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.970624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.971653 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.971657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.972369 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:00.972383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.978493 (+ 6110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.978500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:00.978589 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.979058 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:00.979076 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:00.980584 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:00.985597 (+ 5013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.985601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.985603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.985833 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.986121 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:00.986124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:00.986126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:00.986209 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:00.986635 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:00.986638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:00.986722 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 05:11:00.986815 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:11:00.987001 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 05:11:00.987648 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.968976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.30.88.252:33513)
0811 05:11:00.970099 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.970103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.970115 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:00.970168 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.970170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.970211 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.970735 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.970823 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.971528 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.977933 (+ 6405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.978341 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.978933 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.983066 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.983069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.983071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.983093 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.985504 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.985942 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.985945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.985947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.986022 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.986318 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.986321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.986322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.986526 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.987316 (+ 790us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.987405 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.987517 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048}
I20260811 05:11:00.987818 14379 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:11:00.988299 14406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:11:00.988370 14406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:11:00.985921 14221 master_runner.cc:186] Error getting master registration for 127.30.88.254:43589: OK, instance_id { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" instance_seqno: 1786425060508224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:11:00.990620 14221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 05:11:00.993009 14420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:11:00.993158 14420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:11:00.991676 14305 master_runner.cc:186] Error getting master registration for 127.30.88.254:43589: OK, instance_id { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" instance_seqno: 1786425060508224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:11:00.997478 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: Bootstrap starting.
I20260811 05:11:01.000197 14213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:01.000454 14305 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:11:00.995661 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:00.972017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.30.88.252:33513)
0811 05:11:00.977989 (+ 5972us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:00.977993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:00.981359 (+ 3366us) server_negotiation.cc:407] Connection header received
0811 05:11:00.981395 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:00.981397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:00.981438 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:00.981504 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:00.982246 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.982907 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.984156 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:00.984752 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:00.984844 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:00.992021 (+ 7177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:00.992025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:00.992027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:00.992049 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:00.993652 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.994131 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.994137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.994139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.994216 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:00.994484 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:00.994486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:00.994488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:00.994667 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:00.994775 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:00.995432 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 05:11:00.995533 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5901}
I20260811 05:11:00.999050 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.000900 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:01.001801 14213 log.cc:824] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:01.004681 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: No bootstrap required, opened a new log
I20260811 05:11:01.010135 14213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:01.010346 14213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:01.010447 14213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4f82eb4ada429c92d65eac3df85ad6, State: Initialized, Role: FOLLOWER
I20260811 05:11:01.011032 14213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:01.011505 14423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.011689 14423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:01.013283 14213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:11:01.017930 14438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:11:01.018050 14438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:11:01.022603 14139 master_runner.cc:263] No leader master found from master 7e4f82eb4ada429c92d65eac3df85ad6
I20260811 05:11:01.022665 14139 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 05:11:01.161948 14327 tablet.cc:2369] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:01.171466 14408 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:01.171658 14408 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:01.172745 14408 leader_election.cc:288] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e4f82eb4ada429c92d65eac3df85ad6 (127.30.88.254:43589), 5ef07ba69640423c9c7dee3de6cb6d4c (127.30.88.253:36375)
I20260811 05:11:01.173182 14183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac6537e8203d4c9a8ecf033a5c174017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" is_pre_election: true
I20260811 05:11:01.173295 14270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac6537e8203d4c9a8ecf033a5c174017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" is_pre_election: true
I20260811 05:11:01.173484 14183 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac6537e8203d4c9a8ecf033a5c174017 in term 0.
I20260811 05:11:01.173583 14270 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac6537e8203d4c9a8ecf033a5c174017 in term 0.
I20260811 05:11:01.173858 14324 leader_election.cc:302] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e4f82eb4ada429c92d65eac3df85ad6, ac6537e8203d4c9a8ecf033a5c174017; no voters:
I20260811 05:11:01.174446 14408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:11:01.174537 14408 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:11:01.174578 14408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:01.176023 14408 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:01.176386 14408 leader_election.cc:288] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [CANDIDATE]: Term 1 election: Requested vote from peers 7e4f82eb4ada429c92d65eac3df85ad6 (127.30.88.254:43589), 5ef07ba69640423c9c7dee3de6cb6d4c (127.30.88.253:36375)
I20260811 05:11:01.176818 14270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac6537e8203d4c9a8ecf033a5c174017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c"
I20260811 05:11:01.176976 14183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac6537e8203d4c9a8ecf033a5c174017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e4f82eb4ada429c92d65eac3df85ad6"
I20260811 05:11:01.176971 14270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:01.177093 14183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:01.178489 14270 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac6537e8203d4c9a8ecf033a5c174017 in term 1.
I20260811 05:11:01.178871 14183 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac6537e8203d4c9a8ecf033a5c174017 in term 1.
I20260811 05:11:01.178972 14322 leader_election.cc:302] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ef07ba69640423c9c7dee3de6cb6d4c, ac6537e8203d4c9a8ecf033a5c174017; no voters:
I20260811 05:11:01.179206 14408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:01.179543 14408 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [term 1 LEADER]: Becoming Leader. State: Replica: ac6537e8203d4c9a8ecf033a5c174017, State: Running, Role: LEADER
I20260811 05:11:01.180051 14408 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } }
I20260811 05:11:01.182018 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6537e8203d4c9a8ecf033a5c174017. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.182140 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:01.182585 14442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:01.185535 14442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:01.193638 14270 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [term 1 FOLLOWER]: Refusing update from remote peer ac6537e8203d4c9a8ecf033a5c174017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:01.193722 14183 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [term 1 FOLLOWER]: Refusing update from remote peer ac6537e8203d4c9a8ecf033a5c174017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:01.194271 14440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:01.194418 14408 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:01.197973 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6537e8203d4c9a8ecf033a5c174017. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.198102 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:01.198338 14423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6537e8203d4c9a8ecf033a5c174017. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.198477 14423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:11:01.200572 14246 tablet.cc:2369] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:01.201112 14443 mvcc.cc:205] Tried to move back new op lower bound from 7317197050641969152 to 7317197050597240832. Current Snapshot: MvccSnapshot[applied={T|T < 7317197050641969152}]
I20260811 05:11:01.201246 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.201339 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:01.202358 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.202450 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:01.203744 14445 mvcc.cc:205] Tried to move back new op lower bound from 7317197050641969152 to 7317197050597240832. Current Snapshot: MvccSnapshot[applied={T|T < 7317197050641969152}]
I20260811 05:11:01.204455 14442 catalog_manager.cc:1347] Generated new cluster ID: c5c17204559945edbcdae112a4448a13
I20260811 05:11:01.204555 14442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:01.206986 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.207100 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:01.208163 14423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac6537e8203d4c9a8ecf033a5c174017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e4f82eb4ada429c92d65eac3df85ad6" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43589 } } peers { permanent_uuid: "5ef07ba69640423c9c7dee3de6cb6d4c" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 36375 } } peers { permanent_uuid: "ac6537e8203d4c9a8ecf033a5c174017" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 33513 } } }
I20260811 05:11:01.208287 14423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:11:01.236335 14442 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:11:01.237531 14442 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:01.245010 14442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017: Generated new TSK 0
I20260811 05:11:01.245762 14442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:11:01.989871 14406 catalog_manager.cc:1259] Loaded cluster ID: c5c17204559945edbcdae112a4448a13
I20260811 05:11:01.989965 14406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: loading cluster ID for follower catalog manager: success
I20260811 05:11:01.991906 14406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: acquiring CA information for follower catalog manager: success
I20260811 05:11:01.992969 14406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ef07ba69640423c9c7dee3de6cb6d4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:11:02.019294 14438 catalog_manager.cc:1259] Loaded cluster ID: c5c17204559945edbcdae112a4448a13
I20260811 05:11:02.019354 14438 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: loading cluster ID for follower catalog manager: success
I20260811 05:11:02.020830 14438 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: acquiring CA information for follower catalog manager: success
I20260811 05:11:02.021881 14438 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e4f82eb4ada429c92d65eac3df85ad6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:11:02.299810 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14139
I20260811 05:11:02.309181 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14221
I20260811 05:11:02.315089 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:02.314700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43589 (local address 127.0.0.1:33734)
0811 05:11:02.314948 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:02.315009 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
W20260811 05:11:02.315274 14324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.254:43589: connect: Connection refused (error 111)
I20260811 05:11:02.316156 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:02.315887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:43589 (local address 127.0.0.1:33748)
0811 05:11:02.316023 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:02.316064 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:43589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:11:02.316412 14324 consensus_peers.cc:487] T 00000000000000000000000000000000 P ac6537e8203d4c9a8ecf033a5c174017 -> Peer 7e4f82eb4ada429c92d65eac3df85ad6 (127.30.88.254:43589): Couldn't send request to peer 7e4f82eb4ada429c92d65eac3df85ad6. Status: Network error: Client connection negotiation failed: client connection to 127.30.88.254:43589: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:11:02.316566 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14305
2026-08-11T05:11:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3163 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:11:02 dist-test-slave-7c35 krb5kdc[14461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:11:02 dist-test-slave-7c35 krb5kdc[14461](info): set up 2 sockets
Aug 11 05:11:02 dist-test-slave-7c35 krb5kdc[14461](info): commencing operation
krb5kdc: starting...
W20260811 05:11:03.385913 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:11:03.505127 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34013
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:34013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:03.615483 14474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:03.615756 14474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:03.615857 14474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:03.622344 14474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:03.622424 14474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:03.622464 14474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:03.622505 14474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:03.622543 14474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:03.628455 14474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:34013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:03.629495 14474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:03.630501 14474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:03.638880 14479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:03.638914 14480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:03.638984 14482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:03.639091 14474 server_base.cc:1034] running on GCE node
I20260811 05:11:03.639743 14474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:03.640812 14474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:03.641996 14474 hybrid_clock.cc:648] HybridClock initialized: now 1786425063641985 us; error 38 us; skew 500 ppm
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425063, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:03.644753 14474 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:03.645542 14474 webserver.cc:460] Webserver started at http://127.30.88.254:45925/ using document root <none> and password file <none>
I20260811 05:11:03.646096 14474 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:03.646185 14474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:03.646430 14474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:03.648478 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "59bb42efeb4c4944bec0c771fd0001cb"
format_stamp: "Formatted at 2026-08-11 05:11:03 on dist-test-slave-7c35"
server_key: "5e636ce048c0b23e0777d7bf7c8ebd22"
server_key_iv: "d87e6e10a3639437cb4a5503d2d62d34"
server_key_version: "encryptionkey@0"
I20260811 05:11:03.649046 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "59bb42efeb4c4944bec0c771fd0001cb"
format_stamp: "Formatted at 2026-08-11 05:11:03 on dist-test-slave-7c35"
server_key: "5e636ce048c0b23e0777d7bf7c8ebd22"
server_key_iv: "d87e6e10a3639437cb4a5503d2d62d34"
server_key_version: "encryptionkey@0"
I20260811 05:11:03.652807 14474 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 05:11:03.655194 14489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:03.656385 14474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:03.656525 14474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "59bb42efeb4c4944bec0c771fd0001cb"
format_stamp: "Formatted at 2026-08-11 05:11:03 on dist-test-slave-7c35"
server_key: "5e636ce048c0b23e0777d7bf7c8ebd22"
server_key_iv: "d87e6e10a3639437cb4a5503d2d62d34"
server_key_version: "encryptionkey@0"
I20260811 05:11:03.656646 14474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:03.667433 14474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:03.670004 14474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:03.670226 14474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:03.677079 14474 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:34013
I20260811 05:11:03.677139 14551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:34013 every 8 connection(s)
I20260811 05:11:03.678073 14474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:03.680707 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14474
I20260811 05:11:03.680805 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:11:03.680894 14552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:03.681164 31075 external_mini_cluster.cc:1428] Setting key 744946ca62ea98142d5dfd9556a49708
I20260811 05:11:03.686693 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb: Bootstrap starting.
I20260811 05:11:03.688832 14552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb: Neither blocks nor log segments found. Creating new log.
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:03.689743 14552 log.cc:824] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:03.691845 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb: No bootstrap required, opened a new log
I20260811 05:11:03.694576 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:03.682473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.30.88.254:34013)
0811 05:11:03.682942 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:03.682956 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:03.682988 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:11:03.683547 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:03.683563 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:03.683845 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:03.684154 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:03.684911 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:03.685805 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:03.686622 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:03.687157 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:03.687327 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:03.690076 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:03.690091 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:03.690104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:03.690156 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:03.691935 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:03.692379 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:03.692384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:03.692388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:03.692493 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:03.692782 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:03.692786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:03.692788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:03.693187 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:03.693382 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:03.693662 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 05:11:03.693944 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":142,"threads_started":1}
I20260811 05:11:03.695305 14552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } }
I20260811 05:11:03.695554 14552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:03.695611 14552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59bb42efeb4c4944bec0c771fd0001cb, State: Initialized, Role: FOLLOWER
I20260811 05:11:03.696063 14552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } }
I20260811 05:11:03.696185 14552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:03.696262 14552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:03.696363 14552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:03.697351 14552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } }
I20260811 05:11:03.697435 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:11:03.697675 14552 leader_election.cc:302] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59bb42efeb4c4944bec0c771fd0001cb; no voters:
I20260811 05:11:03.697991 14552 leader_election.cc:288] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:11:03.698071 14557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:03.698256 14557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [term 1 LEADER]: Becoming Leader. State: Replica: 59bb42efeb4c4944bec0c771fd0001cb, State: Running, Role: LEADER
I20260811 05:11:03.698563 14557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } }
I20260811 05:11:03.699231 14552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:03.700306 14558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59bb42efeb4c4944bec0c771fd0001cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } } }
I20260811 05:11:03.700306 14559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59bb42efeb4c4944bec0c771fd0001cb. Latest consensus state: current_term: 1 leader_uuid: "59bb42efeb4c4944bec0c771fd0001cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59bb42efeb4c4944bec0c771fd0001cb" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34013 } } }
I20260811 05:11:03.700461 14559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:03.700461 14558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:03.700907 14566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:03.703908 14566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:03.709103 14566 catalog_manager.cc:1347] Generated new cluster ID: 53ff4eb469784a52a8a9f6e59e305c90
I20260811 05:11:03.709177 14566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:03.717327 14566 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:11:03.718130 14566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:03.722880 14566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59bb42efeb4c4944bec0c771fd0001cb: Generated new TSK 0
I20260811 05:11:03.723409 14566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:11:03.753347 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14474
2026-08-11T05:11:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1438 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14582](info): set up 2 sockets
Aug 11 05:11:03 dist-test-slave-7c35 krb5kdc[14582](info): commencing operation
krb5kdc: starting...
W20260811 05:11:04.806419 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:11:04 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:11:04.918044 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:39199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:05.022928 14597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:05.023157 14597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:05.023192 14597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:05.027096 14597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:05.027137 14597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:05.027153 14597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:05.027168 14597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:05.027181 14597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:05.032478 14597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:39199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:05.033377 14597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:05.034205 14597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:05.042158 14603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.042200 14602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.042460 14597 server_base.cc:1034] running on GCE node
W20260811 05:11:05.042173 14605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.043138 14597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:05.044159 14597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:05.045351 14597 hybrid_clock.cc:648] HybridClock initialized: now 1786425065045315 us; error 48 us; skew 500 ppm
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:05.047946 14597 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:05.048765 14597 webserver.cc:460] Webserver started at http://127.30.88.254:39621/ using document root <none> and password file <none>
I20260811 05:11:05.049295 14597 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:05.049387 14597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:05.049623 14597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:05.051695 14597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "1d639b3fce16408e8ce797dd5d7a54a5"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "3e2f05f6444f473af92cee0562a504bf"
server_key_iv: "4cdd13985ad458778469753d9f354dec"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.052248 14597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "1d639b3fce16408e8ce797dd5d7a54a5"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "3e2f05f6444f473af92cee0562a504bf"
server_key_iv: "4cdd13985ad458778469753d9f354dec"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.055552 14597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:11:05.057734 14612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.059000 14597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:11:05.059128 14597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1d639b3fce16408e8ce797dd5d7a54a5"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "3e2f05f6444f473af92cee0562a504bf"
server_key_iv: "4cdd13985ad458778469753d9f354dec"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.059243 14597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:05.099884 14597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:05.102394 14597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:05.102598 14597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:05.108978 14597 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:39199
I20260811 05:11:05.109246 14667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:39199 every 8 connection(s)
I20260811 05:11:05.109963 14597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:05.112676 14668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:05.114032 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14597
I20260811 05:11:05.114154 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:11:05.114445 31075 external_mini_cluster.cc:1428] Setting key 14052fdc6e656d10d306c42f488f2e95
I20260811 05:11:05.118829 14668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap starting.
I20260811 05:11:05.121492 14668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:05.122347 14668 log.cc:824] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:05.124092 14668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: No bootstrap required, opened a new log
I20260811 05:11:05.127267 14668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:05.127492 14668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:05.127563 14668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Initialized, Role: FOLLOWER
I20260811 05:11:05.128047 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.115685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.30.88.254:39199)
0811 05:11:05.116036 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.116050 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.116082 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:11:05.116665 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.116680 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.116960 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.117228 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:05.118149 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.119197 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.120389 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.120831 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.120941 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.123619 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:05.123624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:05.123635 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:05.123657 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:05.125254 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.125708 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.125714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.125726 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.125833 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.126120 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.126124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.126125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.126503 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:05.126714 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.126976 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.127239 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":113,"threads_started":1}
I20260811 05:11:05.128050 14668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:05.128266 14668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:05.128342 14668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:05.128433 14668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:05.129380 14668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:05.129721 14668 leader_election.cc:302] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d639b3fce16408e8ce797dd5d7a54a5; no voters:
I20260811 05:11:05.130031 14668 leader_election.cc:288] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:11:05.130139 14674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:05.130383 14674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 LEADER]: Becoming Leader. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Running, Role: LEADER
I20260811 05:11:05.130736 14674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:05.131266 14668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:05.131831 14677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d639b3fce16408e8ce797dd5d7a54a5. Latest consensus state: current_term: 1 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:05.132001 14677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:05.132226 14675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:05.132318 14675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:05.132400 14686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:05.135387 14686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:05.140911 14686 catalog_manager.cc:1347] Generated new cluster ID: c24c17a8003846958ecf3125274c361b
I20260811 05:11:05.140977 14686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:05.161415 14686 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:11:05.162554 14686 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:05.174134 14686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Generated new TSK 0
I20260811 05:11:05.174713 14686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:11:05.210111 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:05.322961 14703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:05.323161 14703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:05.323196 14703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:05.326565 14703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:05.326606 14703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:05.326650 14703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:11:05.331563 14703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:05.332414 14703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:05.333369 14703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:11:05.342458 14703 server_base.cc:1034] running on GCE node
W20260811 05:11:05.342422 14709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.342422 14708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.342427 14711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.343101 14703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:05.343658 14703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:05.344851 14703 hybrid_clock.cc:648] HybridClock initialized: now 1786425065344829 us; error 44 us; skew 500 ppm
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:05.347474 14703 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:11:05.348234 14703 webserver.cc:460] Webserver started at http://127.30.88.193:43785/ using document root <none> and password file <none>
I20260811 05:11:05.348760 14703 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:05.348848 14703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:05.349078 14703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:05.350962 14703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "5c190330708f40088a613734f31fc185"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "24949c5154634a70173caa3d68375c19"
server_key_iv: "5cd01421b707689566b01695e7469e95"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.351569 14703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "5c190330708f40088a613734f31fc185"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "24949c5154634a70173caa3d68375c19"
server_key_iv: "5cd01421b707689566b01695e7469e95"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.355089 14703 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 05:11:05.357416 14718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.358451 14703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:05.358582 14703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "5c190330708f40088a613734f31fc185"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "24949c5154634a70173caa3d68375c19"
server_key_iv: "5cd01421b707689566b01695e7469e95"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.358702 14703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:05.374189 14703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:05.376797 14703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:05.376982 14703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:05.377507 14703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:05.378276 14703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:05.378346 14703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.378413 14703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:05.378465 14703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.386933 14703 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:40731
I20260811 05:11:05.386952 14844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:40731 every 8 connection(s)
I20260811 05:11:05.387683 14703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:11:05.396215 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14703
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:05.396351 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:11:05.396682 31075 external_mini_cluster.cc:1428] Setting key 0ebeb67b7e49605a3d168017421d7633
I20260811 05:11:05.400753 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.389272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:55669 (local address 127.30.88.254:39199)
0811 05:11:05.389418 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.389423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.390276 (+ 853us) server_negotiation.cc:407] Connection header received
0811 05:11:05.391618 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.391621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.391664 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.391733 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:05.392782 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.393297 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.394292 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.394489 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.394558 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.396994 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:05.396997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:05.396999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:05.397023 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:05.398695 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.399259 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.399265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.399270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.399374 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.399703 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.399709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.399713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.399967 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:05.400066 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.400543 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.400643 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:11:05.401758 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.389615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:55669)
0811 05:11:05.390077 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.390157 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.391441 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.391880 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.391885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.392231 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:05.392658 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.392671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.393411 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.393413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.394195 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.394204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.394604 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.394606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.394745 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.395239 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:05.395253 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:05.396819 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:05.398833 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.398838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.398887 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.399152 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.399473 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.399476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.399477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.399601 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.400063 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:05.400068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:05.400278 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.400477 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.400790 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260811 05:11:05.403015 14845 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:05.403250 14845 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:05.403762 14845 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
I20260811 05:11:05.405318 14631 ts_manager.cc:194] Registered new tserver with Master: 5c190330708f40088a613734f31fc185 (127.30.88.193:40731)
I20260811 05:11:05.406769 14631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:55669
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:11:05.424897 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:05.526268 14851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:05.526455 14851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:05.526489 14851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:05.530222 14851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:05.530270 14851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:05.530357 14851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:11:05.535264 14851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:05.536062 14851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:05.536902 14851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:05.545987 14856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.546062 14851 server_base.cc:1034] running on GCE node
W20260811 05:11:05.546195 14859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.546205 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.546677 14851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:05.547283 14851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:05.548444 14851 hybrid_clock.cc:648] HybridClock initialized: now 1786425065548427 us; error 30 us; skew 500 ppm
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:05.551120 14851 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:11:05.551954 14851 webserver.cc:460] Webserver started at http://127.30.88.194:45691/ using document root <none> and password file <none>
I20260811 05:11:05.552474 14851 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:05.552532 14851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:05.552750 14851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:05.554565 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "7678cdbca2e84ec5808bc8ce28428561"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "9a98500da5d5cee014741b304f74e7cb"
server_key_iv: "cc4a0028853fc57ff035e82397b526b2"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.555184 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "7678cdbca2e84ec5808bc8ce28428561"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "9a98500da5d5cee014741b304f74e7cb"
server_key_iv: "cc4a0028853fc57ff035e82397b526b2"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.558710 14851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 05:11:05.560905 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.562009 14851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:05.562119 14851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "7678cdbca2e84ec5808bc8ce28428561"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "9a98500da5d5cee014741b304f74e7cb"
server_key_iv: "cc4a0028853fc57ff035e82397b526b2"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.562249 14851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:05.594162 14851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:05.597074 14851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:05.597286 14851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:05.597937 14851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:05.598780 14851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:05.598829 14851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.598930 14851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:05.598968 14851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.607584 14851 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:43485
I20260811 05:11:05.607612 14990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:43485 every 8 connection(s)
I20260811 05:11:05.608439 14851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:11:05.610416 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14851
I20260811 05:11:05.610534 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:11:05.610891 31075 external_mini_cluster.cc:1428] Setting key b0b27a278fffe4ca3e5e311a655ecde1
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:05.625238 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.610207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:47357 (local address 127.30.88.254:39199)
0811 05:11:05.610338 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.610346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.611388 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 05:11:05.612952 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.612963 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.613015 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.613086 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:05.614295 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.615200 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.616918 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.617290 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.617380 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.620726 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:05.620729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:05.620732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:05.620759 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:05.622382 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.623120 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.623126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.623130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.623249 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.623820 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.623827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.623831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.624080 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:05.624182 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.624932 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.625069 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:11:05.626847 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.610675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:47357)
0811 05:11:05.611222 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.611262 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.612760 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.613239 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.613245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.613556 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:05.614092 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.614132 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.615368 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.615372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.616780 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.616791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.617606 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.617612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.617823 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.618433 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:05.618470 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:05.620550 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:05.622534 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.622543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.622561 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.622995 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.623379 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.623406 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.623411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.623671 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.624261 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:05.624269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:05.624606 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.625467 (+ 861us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.625834 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":169,"threads_started":1}
I20260811 05:11:05.628198 14991 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:05.628538 14991 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:05.629182 14991 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:11:05.630601 14631 ts_manager.cc:194] Registered new tserver with Master: 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485)
I20260811 05:11:05.631402 14631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:47357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:11:05.649242 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:05.752655 14996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:05.752848 14996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:05.752883 14996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:05.756479 14996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:05.756522 14996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:05.756575 14996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:11:05.761112 14996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:05.762001 14996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:05.762893 14996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:05.771498 15003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.771515 15006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:05.771513 15004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:05.771548 14996 server_base.cc:1034] running on GCE node
I20260811 05:11:05.772141 14996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:05.772652 14996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:05.773813 14996 hybrid_clock.cc:648] HybridClock initialized: now 1786425065773792 us; error 38 us; skew 500 ppm
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:05.776407 14996 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:11:05.777249 14996 webserver.cc:460] Webserver started at http://127.30.88.195:37245/ using document root <none> and password file <none>
I20260811 05:11:05.777762 14996 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:05.777848 14996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:05.778072 14996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:05.780052 14996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "f044e811a7a841b4b402e9bedc5c2428"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "231c3040c93dc68aef8d11a6ebae2f00"
server_key_iv: "da8cd9068a420628bea87acacbfa4bfb"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.780592 14996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "f044e811a7a841b4b402e9bedc5c2428"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "231c3040c93dc68aef8d11a6ebae2f00"
server_key_iv: "da8cd9068a420628bea87acacbfa4bfb"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.784014 14996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:11:05.786144 15013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.787237 14996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:05.787358 14996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f044e811a7a841b4b402e9bedc5c2428"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "231c3040c93dc68aef8d11a6ebae2f00"
server_key_iv: "da8cd9068a420628bea87acacbfa4bfb"
server_key_version: "encryptionkey@0"
I20260811 05:11:05.787454 14996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:05.809043 14996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:05.812317 14996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:05.812500 14996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:05.812999 14996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:05.813726 14996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:05.813767 14996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.813824 14996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:05.813853 14996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:05.821803 14996 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:46757
I20260811 05:11:05.821841 15139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:46757 every 8 connection(s)
I20260811 05:11:05.822571 14996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:11:05.825484 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 14996
I20260811 05:11:05.825627 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:11:05.825990 31075 external_mini_cluster.cc:1428] Setting key 09361a6ae317eca0c5a73b8cc184052a
Aug 11 05:11:05 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:05.835405 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.824244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:37851 (local address 127.30.88.254:39199)
0811 05:11:05.824392 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.824397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.825203 (+ 806us) server_negotiation.cc:407] Connection header received
0811 05:11:05.826894 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.826897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.826967 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.827028 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:05.828299 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.828862 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.829947 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.830176 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.830249 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.832583 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:05.832585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:05.832587 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:05.832608 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:05.833823 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.834241 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.834243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.834244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.834308 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.834593 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.834595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.834596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.834735 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:05.834800 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.835222 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.835305 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:11:05.836021 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.824611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:37851)
0811 05:11:05.825055 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.825101 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.826635 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.827229 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.827235 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.827606 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:05.828172 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.828187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.828965 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.828968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.829832 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.829841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.830293 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.830296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.830429 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.830919 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:05.830934 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:05.832450 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:05.833915 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.833919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.833929 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.834151 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.834377 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:05.834379 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:05.834398 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:05.834510 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:05.834812 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:05.834816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:05.835038 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.835176 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.835371 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":118,"threads_started":1}
I20260811 05:11:05.837029 15140 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:05.837250 15140 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:05.837740 15140 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
I20260811 05:11:05.838773 14631 ts_manager.cc:194] Registered new tserver with Master: f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:05.839538 14631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:37851
I20260811 05:11:05.839923 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:11:05.847832 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.841886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.30.88.254:39199)
0811 05:11:05.842007 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.842009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.842183 (+ 174us) server_negotiation.cc:407] Connection header received
0811 05:11:05.842266 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.842268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.842297 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.842355 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:05.842941 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.843403 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.844114 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.844302 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.844378 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.845515 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:05.845517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:05.845519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:05.845539 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:05.846642 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.847035 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.847038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.847038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.847102 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:05.847289 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:05.847291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:05.847291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:05.847442 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:05.847524 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.847674 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.847746 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:11:05.850479 14631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:11:05.853315 14631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:11:05.866488 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.861294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:46757 (local address 127.0.0.1:55430)
0811 05:11:05.861759 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.861769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.861879 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.862514 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.862517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.862535 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:05.862659 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.862666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.864035 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.864041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.865189 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.865206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.866020 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.866024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.866122 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866157 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.866268 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.866384 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":17,"spinlock_wait_cycles":4608,"thread_start_us":78,"threads_started":1}
I20260811 05:11:05.866825 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.860814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40731 (local address 127.0.0.1:44080)
0811 05:11:05.861721 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.861734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.861878 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.862368 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.862374 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.862398 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:05.862553 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.862558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.863616 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.863622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.864940 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.864952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.866250 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.866253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.866453 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866528 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.866598 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.866680 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":216,"spinlock_wait_cycles":1792,"thread_start_us":85,"threads_started":1}
I20260811 05:11:05.866957 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.860829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.30.88.194:43485)
0811 05:11:05.861207 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.861219 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.861544 (+ 325us) server_negotiation.cc:407] Connection header received
0811 05:11:05.861849 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.861855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.862032 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.862166 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:05.862659 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.863772 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.865188 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.865932 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.866062 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866176 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.866680 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.866777 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":86,"threads_started":1}
I20260811 05:11:05.867115 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.861410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.30.88.195:46757)
0811 05:11:05.862115 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.862124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.862140 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:11:05.862188 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.862191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.862376 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.862882 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:05.863016 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.863912 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.865311 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.865838 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.865985 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866162 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.866892 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.866988 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":116,"threads_started":1}
I20260811 05:11:05.867322 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.861369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.30.88.193:40731)
0811 05:11:05.861623 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:05.861636 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:05.861981 (+ 345us) server_negotiation.cc:407] Connection header received
0811 05:11:05.862057 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:05.862062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:05.862222 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:05.862354 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:05.862679 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.863504 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.865088 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.865932 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.866040 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866176 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:05.867093 (+ 917us) server_negotiation.cc:299] Negotiation successful
0811 05:11:05.867207 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260811 05:11:05.867638 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:05.860719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43485 (local address 127.0.0.1:42852)
0811 05:11:05.861416 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:05.861457 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:05.861644 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:05.862368 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:05.862374 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:05.862403 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:05.862543 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.862549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.863893 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.863895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.865069 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:05.865077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:05.866250 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:05.866253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:05.866454 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:05.866528 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 05:11:05.866599 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:11:05.867512 (+ 913us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":960,"spinlock_wait_cycles":4864,"thread_start_us":66,"threads_started":1}
I20260811 05:11:05.869536 15067 tablet_service.cc:1467] Processing CreateTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 (DEFAULT_TABLE table=test-table [id=72c28132df3b4034ba309b68468bc15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:05.869613 14772 tablet_service.cc:1467] Processing CreateTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 (DEFAULT_TABLE table=test-table [id=72c28132df3b4034ba309b68468bc15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:05.870474 14772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90737b03d9dc454d8e8fe2daf1a83268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:05.870478 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90737b03d9dc454d8e8fe2daf1a83268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:05.876071 15160 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Bootstrap starting.
I20260811 05:11:05.877333 14920 tablet_service.cc:1467] Processing CreateTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 (DEFAULT_TABLE table=test-table [id=72c28132df3b4034ba309b68468bc15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:05.878225 14920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90737b03d9dc454d8e8fe2daf1a83268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:05.878654 15160 tablet_bootstrap.cc:654] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:05.879823 15160 log.cc:824] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:05.880554 15161 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Bootstrap starting.
I20260811 05:11:05.881940 15160 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: No bootstrap required, opened a new log
I20260811 05:11:05.882153 15160 ts_tablet_manager.cc:1397] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:11:05.883065 15161 tablet_bootstrap.cc:654] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:05.883747 15161 log.cc:824] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:05.885030 15165 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Bootstrap starting.
I20260811 05:11:05.885715 15161 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: No bootstrap required, opened a new log
I20260811 05:11:05.885916 15161 ts_tablet_manager.cc:1397] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:11:05.886162 15160 raft_consensus.cc:348] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.886392 15160 raft_consensus.cc:374] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:05.886472 15160 raft_consensus.cc:729] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c190330708f40088a613734f31fc185, State: Initialized, Role: FOLLOWER
I20260811 05:11:05.887004 15160 consensus_queue.cc:260] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.887600 15165 tablet_bootstrap.cc:654] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:05.887652 14845 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
I20260811 05:11:05.888027 15160 ts_tablet_manager.cc:1428] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:11:05.888502 15165 log.cc:824] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Log is configured to *not* fsync() on all Append() calls
W20260811 05:11:05.888648 14846 tablet.cc:2369] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:05.890043 15161 raft_consensus.cc:348] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.890333 15161 raft_consensus.cc:374] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:05.890386 15161 raft_consensus.cc:729] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f044e811a7a841b4b402e9bedc5c2428, State: Initialized, Role: FOLLOWER
I20260811 05:11:05.890616 15165 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: No bootstrap required, opened a new log
I20260811 05:11:05.890789 15165 ts_tablet_manager.cc:1397] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:11:05.890877 15161 consensus_queue.cc:260] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.891629 15140 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
I20260811 05:11:05.891892 15161 ts_tablet_manager.cc:1428] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:11:05.894124 15165 raft_consensus.cc:348] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.894335 15165 raft_consensus.cc:374] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:05.894413 15165 raft_consensus.cc:729] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7678cdbca2e84ec5808bc8ce28428561, State: Initialized, Role: FOLLOWER
I20260811 05:11:05.894814 15165 consensus_queue.cc:260] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:05.895444 14991 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
I20260811 05:11:05.895689 15165 ts_tablet_manager.cc:1428] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:11:06.023011 15169 raft_consensus.cc:482] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:06.023190 15169 raft_consensus.cc:504] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:06.024271 15169 leader_election.cc:288] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c190330708f40088a613734f31fc185 (127.30.88.193:40731), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:06.028034 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.024525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40731 (local address 127.30.88.194:38759)
0811 05:11:06.024989 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.025002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.025110 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.025368 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.025371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.025387 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:06.025490 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.025495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.026188 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.026190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.026876 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.026884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.027478 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.027481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.027681 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.027704 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.027777 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.027830 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":33,"thread_start_us":148,"threads_started":1}
I20260811 05:11:06.028048 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.024794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:38759 (local address 127.30.88.193:40731)
0811 05:11:06.024919 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.024925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.025104 (+ 179us) server_negotiation.cc:407] Connection header received
0811 05:11:06.025187 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.025190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.025239 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.025333 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:06.025595 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.026074 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.027022 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.027362 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.027485 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.027659 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.027818 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.027911 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:11:06.028168 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.024618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:49113 (local address 127.30.88.195:46757)
0811 05:11:06.024764 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.024785 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.024801 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:11:06.024963 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.024967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.025022 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.025097 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:06.025397 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.025965 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.026989 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.027361 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.027482 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.027538 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.027944 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.028016 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:11:06.028079 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.024520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:46757 (local address 127.30.88.194:49113)
0811 05:11:06.024726 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.024740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.024842 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.025126 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.025131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.025152 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:06.025278 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.025289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.026068 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.026071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.026815 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.026824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.027485 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.027488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.027770 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.027803 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.027896 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.027963 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":114}
I20260811 05:11:06.028927 14797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90737b03d9dc454d8e8fe2daf1a83268" candidate_uuid: "7678cdbca2e84ec5808bc8ce28428561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c190330708f40088a613734f31fc185" is_pre_election: true
I20260811 05:11:06.029026 15091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90737b03d9dc454d8e8fe2daf1a83268" candidate_uuid: "7678cdbca2e84ec5808bc8ce28428561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f044e811a7a841b4b402e9bedc5c2428" is_pre_election: true
I20260811 05:11:06.029255 15091 raft_consensus.cc:2454] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7678cdbca2e84ec5808bc8ce28428561 in term 0.
I20260811 05:11:06.029256 14797 raft_consensus.cc:2454] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7678cdbca2e84ec5808bc8ce28428561 in term 0.
I20260811 05:11:06.029754 14867 leader_election.cc:302] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c190330708f40088a613734f31fc185, 7678cdbca2e84ec5808bc8ce28428561; no voters:
I20260811 05:11:06.030061 15169 raft_consensus.cc:2781] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:11:06.030181 15169 raft_consensus.cc:482] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:11:06.030225 15169 raft_consensus.cc:3037] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:06.031432 15169 raft_consensus.cc:504] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:06.031823 15169 leader_election.cc:288] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [CANDIDATE]: Term 1 election: Requested vote from peers 5c190330708f40088a613734f31fc185 (127.30.88.193:40731), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:06.032224 14797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90737b03d9dc454d8e8fe2daf1a83268" candidate_uuid: "7678cdbca2e84ec5808bc8ce28428561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c190330708f40088a613734f31fc185"
I20260811 05:11:06.032292 15091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90737b03d9dc454d8e8fe2daf1a83268" candidate_uuid: "7678cdbca2e84ec5808bc8ce28428561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f044e811a7a841b4b402e9bedc5c2428"
I20260811 05:11:06.032356 14797 raft_consensus.cc:3037] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:06.032431 15091 raft_consensus.cc:3037] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:06.033435 14797 raft_consensus.cc:2454] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7678cdbca2e84ec5808bc8ce28428561 in term 1.
I20260811 05:11:06.033625 15091 raft_consensus.cc:2454] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7678cdbca2e84ec5808bc8ce28428561 in term 1.
I20260811 05:11:06.033918 14871 leader_election.cc:302] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428; no voters:
I20260811 05:11:06.034111 15169 raft_consensus.cc:2781] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:06.034309 15169 raft_consensus.cc:686] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 1 LEADER]: Becoming Leader. State: Replica: 7678cdbca2e84ec5808bc8ce28428561, State: Running, Role: LEADER
I20260811 05:11:06.034621 15169 consensus_queue.cc:237] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:06.037600 14630 catalog_manager.cc:5495] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194). New cstate: current_term: 1 leader_uuid: "7678cdbca2e84ec5808bc8ce28428561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } health_report { overall_health: UNKNOWN } } }
I20260811 05:11:06.053010 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.049298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.30.88.194:43485)
0811 05:11:06.049427 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.049433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.049489 (+ 56us) server_negotiation.cc:407] Connection header received
0811 05:11:06.049662 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.049665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.049726 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.049838 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:06.050109 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.050624 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.051553 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.051779 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.051888 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.052190 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:06.052603 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:11:06.052728 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.052802 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.052854 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:11:06.058377 15091 raft_consensus.cc:1261] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Refusing update from remote peer 7678cdbca2e84ec5808bc8ce28428561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:06.058377 14797 raft_consensus.cc:1261] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Refusing update from remote peer 7678cdbca2e84ec5808bc8ce28428561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:06.059113 15169 consensus_queue.cc:1035] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:06.059335 15171 consensus_queue.cc:1035] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:06.067807 15174 mvcc.cc:205] Tried to move back new op lower bound from 7317197070568714240 to 7317197070480490496. Current Snapshot: MvccSnapshot[applied={T|T < 7317197070568714240 or (T in {7317197070568714240})}]
I20260811 05:11:06.073890 14631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:06.074039 14631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:06.076220 14631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Sending DeleteTablet for 3 replicas of tablet 90737b03d9dc454d8e8fe2daf1a83268
I20260811 05:11:06.076958 14772 tablet_service.cc:1514] Processing DeleteTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:06 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44080
I20260811 05:11:06.077039 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14703
I20260811 05:11:06.077033 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:06 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55430
I20260811 05:11:06.077590 14920 tablet_service.cc:1514] Processing DeleteTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:06 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:42852
I20260811 05:11:06.078500 15184 tablet_replica.cc:330] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: stopping tablet replica
I20260811 05:11:06.078816 15184 raft_consensus.cc:2229] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:11:06.079224 15184 raft_consensus.cc:2258] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:06.079547 15183 tablet_replica.cc:330] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: stopping tablet replica
I20260811 05:11:06.079802 15183 raft_consensus.cc:2229] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:06.080085 15183 raft_consensus.cc:2258] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:06.081235 15184 ts_tablet_manager.cc:1905] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:06.084442 15183 ts_tablet_manager.cc:1905] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:06.087076 15183 ts_tablet_manager.cc:1918] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:06.087167 15183 log.cc:1196] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/90737b03d9dc454d8e8fe2daf1a83268
I20260811 05:11:06.087518 15183 ts_tablet_manager.cc:1939] T 90737b03d9dc454d8e8fe2daf1a83268 P f044e811a7a841b4b402e9bedc5c2428: Deleting consensus metadata
I20260811 05:11:06.088315 15184 ts_tablet_manager.cc:1918] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:06.088426 15184 log.cc:1196] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/90737b03d9dc454d8e8fe2daf1a83268
I20260811 05:11:06.088683 14616 catalog_manager.cc:4841] TS f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): tablet 90737b03d9dc454d8e8fe2daf1a83268 (table test-table [id=72c28132df3b4034ba309b68468bc15a]) successfully deleted
I20260811 05:11:06.088762 15184 ts_tablet_manager.cc:1939] T 90737b03d9dc454d8e8fe2daf1a83268 P 7678cdbca2e84ec5808bc8ce28428561: Deleting consensus metadata
I20260811 05:11:06.089859 14616 catalog_manager.cc:4841] TS 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): tablet 90737b03d9dc454d8e8fe2daf1a83268 (table test-table [id=72c28132df3b4034ba309b68468bc15a]) successfully deleted
W20260811 05:11:06.090983 14614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:40731 (error 108)
I20260811 05:11:06.091231 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14851
I20260811 05:11:06.095408 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.095139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40731 (local address 127.0.0.1:44092)
0811 05:11:06.095275 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.095333 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 05:11:06.095597 14614 catalog_manager.cc:4584] TS 5c190330708f40088a613734f31fc185 (127.30.88.193:40731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90737b03d9dc454d8e8fe2daf1a83268: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40731: connect: Connection refused (error 111)
I20260811 05:11:06.100155 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14996
I20260811 05:11:06.106643 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 14597
I20260811 05:11:06.115157 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--webserver_interface=127.30.88.254
--webserver_port=39621
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:39199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:06.220129 15186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:06.220325 15186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:06.220363 15186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:06.223735 15186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:06.223773 15186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:06.223788 15186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:06.223803 15186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:06.223816 15186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:06.228089 15186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:39199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=39621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:06.228905 15186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:06.229743 15186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:06.242835 15193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.242868 15196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:06.242920 15186 server_base.cc:1034] running on GCE node
W20260811 05:11:06.242832 15194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:06.243633 15186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:06.244628 15186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:06.245783 15186 hybrid_clock.cc:648] HybridClock initialized: now 1786425066245769 us; error 25 us; skew 500 ppm
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:06.248418 15186 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:06.249275 15186 webserver.cc:460] Webserver started at http://127.30.88.254:39621/ using document root <none> and password file <none>
I20260811 05:11:06.249790 15186 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:06.249858 15186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:06.253568 15186 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:06.255537 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.256649 15186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:06.256779 15186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1d639b3fce16408e8ce797dd5d7a54a5"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "3e2f05f6444f473af92cee0562a504bf"
server_key_iv: "4cdd13985ad458778469753d9f354dec"
server_key_version: "encryptionkey@0"
I20260811 05:11:06.257295 15186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:06.278332 15186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:06.281145 15186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:06.281333 15186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:06.288422 15186 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:39199
I20260811 05:11:06.288683 15265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:39199 every 8 connection(s)
I20260811 05:11:06.289523 15186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:06.291170 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15186
I20260811 05:11:06.291622 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:40731
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=43785
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:06.294034 15266 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:11:06.295681 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap starting.
I20260811 05:11:06.311169 15266 log.cc:824] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:06.318785 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:06.319161 15266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap complete.
I20260811 05:11:06.322896 15266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:06.323275 15266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Initialized, Role: FOLLOWER
I20260811 05:11:06.323688 15266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:06.323812 15266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:06.323892 15266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:06.323999 15266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:11:06.325580 15266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:06.325892 15266 leader_election.cc:302] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d639b3fce16408e8ce797dd5d7a54a5; no voters:
I20260811 05:11:06.326195 15266 leader_election.cc:288] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:11:06.326263 15271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:11:06.326440 15271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 LEADER]: Becoming Leader. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Running, Role: LEADER
I20260811 05:11:06.326715 15271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:06.327407 15266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:06.327687 15272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:06.328401 15272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:06.328294 15273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d639b3fce16408e8ce797dd5d7a54a5. Latest consensus state: current_term: 2 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:06.328719 15273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:06.329007 15281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:06.333694 15281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:06.334303 15281 catalog_manager.cc:1259] Loaded cluster ID: c24c17a8003846958ecf3125274c361b
I20260811 05:11:06.334355 15281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:06.335660 15281 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 05:11:06.340049 15281 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 05:11:06.345057 15281 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:06.345669 15281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Loaded TSK: 0
I20260811 05:11:06.346742 15281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:11:06.400933 15268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:06.401167 15268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:06.401217 15268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:06.405352 15268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:06.405402 15268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:06.405560 15268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:11:06.410560 15268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:40731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=43785
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:06.411492 15268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:06.412348 15268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:06.421070 15295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.421059 15294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.421051 15297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:06.421777 15268 server_base.cc:1034] running on GCE node
I20260811 05:11:06.422181 15268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:06.422919 15268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:06.424117 15268 hybrid_clock.cc:648] HybridClock initialized: now 1786425066424089 us; error 36 us; skew 500 ppm
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:06.427392 15268 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:11:06.428196 15268 webserver.cc:460] Webserver started at http://127.30.88.193:43785/ using document root <none> and password file <none>
I20260811 05:11:06.428756 15268 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:06.428838 15268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:06.432574 15268 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 05:11:06.434671 15305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.435835 15268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:06.435966 15268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "5c190330708f40088a613734f31fc185"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "24949c5154634a70173caa3d68375c19"
server_key_iv: "5cd01421b707689566b01695e7469e95"
server_key_version: "encryptionkey@0"
I20260811 05:11:06.436434 15268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:06.450645 15268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:06.453553 15268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:06.453744 15268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:06.454334 15268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:06.455361 15313 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:11:06.459940 15268 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:11:06.460031 15268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 05:11:06.460091 15268 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:11:06.461642 15268 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:11:06.461702 15268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 05:11:06.461937 15313 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Bootstrap starting.
I20260811 05:11:06.471453 15268 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:40731
I20260811 05:11:06.471506 15433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:40731 every 8 connection(s)
I20260811 05:11:06.472286 15268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:11:06.478551 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15268
I20260811 05:11:06.479074 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:43485
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=45691
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:06.482115 15313 log.cc:824] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:06.489883 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.474399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:36433 (local address 127.30.88.254:39199)
0811 05:11:06.474812 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.474830 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.475163 (+ 333us) server_negotiation.cc:407] Connection header received
0811 05:11:06.476331 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.476354 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.476850 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.477248 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:06.478397 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.480090 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.481595 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.482065 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.482280 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.484952 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:06.484964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:06.484976 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:06.485005 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:06.486710 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.487294 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.487299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.487302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.487414 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.487776 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.487780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.487781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.488110 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:06.488254 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.488797 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.489065 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
I20260811 05:11:06.489897 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.474409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:36433)
0811 05:11:06.474971 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.475022 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.475810 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.477296 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.477302 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.477713 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:06.478188 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.478202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.480273 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.480280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.481458 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.481469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.482293 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.482298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.482516 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.483179 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:06.483195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:06.484748 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:06.486854 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.486887 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.486901 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.487167 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.487523 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.487527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.487529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.487672 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.488228 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:06.488234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:06.488472 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.488640 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.489012 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":129,"threads_started":1}
I20260811 05:11:06.491991 15313 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:06.492435 15313 tablet_bootstrap.cc:492] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Bootstrap complete.
I20260811 05:11:06.493148 15434 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:06.493376 15434 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:06.493858 15434 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
I20260811 05:11:06.493955 15313 ts_tablet_manager.cc:1397] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.004s
I20260811 05:11:06.496421 15221 ts_manager.cc:194] Registered new tserver with Master: 5c190330708f40088a613734f31fc185 (127.30.88.193:40731)
I20260811 05:11:06.496690 15313 raft_consensus.cc:348] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:06.497165 15313 raft_consensus.cc:729] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c190330708f40088a613734f31fc185, State: Initialized, Role: FOLLOWER
I20260811 05:11:06.497597 15313 consensus_queue.cc:260] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } }
I20260811 05:11:06.498484 15313 ts_tablet_manager.cc:1428] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260811 05:11:06.500187 15221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:36433
I20260811 05:11:06.502406 15434 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:11:06.509640 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.499319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40731 (local address 127.0.0.1:44108)
0811 05:11:06.499711 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.499754 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.499867 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.501347 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.501352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.501711 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:06.501819 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.501826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.502942 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.502945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.503832 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.503844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.504355 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.504358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.504458 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.504977 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:06.504991 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:06.506344 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:06.507986 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.507994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.507999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.508202 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.508492 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.508495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.508496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.508576 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.509022 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:06.509028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:06.509149 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.509286 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.509469 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":112,"threads_started":1}
I20260811 05:11:06.509685 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.499401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.30.88.193:40731)
0811 05:11:06.500932 (+ 1531us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.500941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.500958 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:11:06.501005 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.501008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.501172 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.501323 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:06.501942 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.502828 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.503959 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.504256 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.504327 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.506492 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:06.506499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:06.506507 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:06.506525 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:06.507873 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.508314 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.508319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.508320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.508388 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.508667 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.508670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.508671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.508898 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:06.509042 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.509378 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.509541 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1448,"thread_start_us":129,"threads_started":1}
I20260811 05:11:06.510529 15360 tablet_service.cc:1514] Processing DeleteTablet for tablet 90737b03d9dc454d8e8fe2daf1a83268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:06 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44108
I20260811 05:11:06.511325 15445 tablet_replica.cc:330] stopping tablet replica
I20260811 05:11:06.511592 15445 raft_consensus.cc:2229] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:06.511822 15445 raft_consensus.cc:2258] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:06.512585 15445 ts_tablet_manager.cc:1905] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:06.515836 15445 ts_tablet_manager.cc:1918] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:06.515920 15445 log.cc:1196] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/90737b03d9dc454d8e8fe2daf1a83268
I20260811 05:11:06.516211 15445 ts_tablet_manager.cc:1939] T 90737b03d9dc454d8e8fe2daf1a83268 P 5c190330708f40088a613734f31fc185: Deleting consensus metadata
I20260811 05:11:06.517628 15205 catalog_manager.cc:4841] TS 5c190330708f40088a613734f31fc185 (127.30.88.193:40731): tablet 90737b03d9dc454d8e8fe2daf1a83268 (table test-table [id=72c28132df3b4034ba309b68468bc15a]) successfully deleted
W20260811 05:11:06.592309 15439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:06.592561 15439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:06.592641 15439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:06.596282 15439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:06.596367 15439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:06.596450 15439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:11:06.601265 15439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:43485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=45691
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:06.602124 15439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:06.603088 15439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:06.611598 15455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.611610 15452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.611609 15453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:06.611824 15439 server_base.cc:1034] running on GCE node
I20260811 05:11:06.612305 15439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:06.612941 15439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:06.614090 15439 hybrid_clock.cc:648] HybridClock initialized: now 1786425066614074 us; error 27 us; skew 500 ppm
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:06.616660 15439 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:11:06.617609 15439 webserver.cc:460] Webserver started at http://127.30.88.194:45691/ using document root <none> and password file <none>
I20260811 05:11:06.618141 15439 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:06.618212 15439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:06.621672 15439 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:06.623584 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.624714 15439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:06.624825 15439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "7678cdbca2e84ec5808bc8ce28428561"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "9a98500da5d5cee014741b304f74e7cb"
server_key_iv: "cc4a0028853fc57ff035e82397b526b2"
server_key_version: "encryptionkey@0"
I20260811 05:11:06.625313 15439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:06.634696 15439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:06.637629 15439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:06.637754 15439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:06.638247 15439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:06.639117 15439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:06.639163 15439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.639228 15439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:06.639268 15439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.648142 15439 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:43485
I20260811 05:11:06.648165 15594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:43485 every 8 connection(s)
I20260811 05:11:06.649044 15439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:11:06.657624 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15439
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:06.658028 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:46757
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=37245
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:06.662619 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.650763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:60369 (local address 127.30.88.254:39199)
0811 05:11:06.650943 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.650948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.651764 (+ 816us) server_negotiation.cc:407] Connection header received
0811 05:11:06.653044 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.653047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.653105 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.653180 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:06.654266 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.654782 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.655891 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.656125 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.656207 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.658653 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:06.658656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:06.658658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:06.658683 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:06.660399 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.660993 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.660996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.660997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.661107 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.661513 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.661517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.661519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.661725 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:06.661834 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.662281 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.662434 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:11:06.663453 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.651139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:60369)
0811 05:11:06.651572 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.651627 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.652844 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.653348 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.653353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.653695 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:06.654122 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.654135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.654924 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.654926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.655778 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.655787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.656251 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.656253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.656383 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.656970 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:06.656986 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:06.658478 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:06.660536 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.660544 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.660560 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.660895 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.661203 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.661208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.661212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.661382 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.661833 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:06.661839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:06.662053 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.662232 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.662541 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260811 05:11:06.664528 15596 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:06.664759 15596 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:06.665227 15596 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
I20260811 05:11:06.666304 15221 ts_manager.cc:194] Registered new tserver with Master: 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485)
I20260811 05:11:06.667181 15221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:60369
W20260811 05:11:06.764055 15600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:06.764248 15600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:06.764282 15600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:06.767870 15600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:06.767915 15600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:06.767999 15600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:11:06.772676 15600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:46757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=37245
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:06.773465 15600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:06.774290 15600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:06.783190 15609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.783210 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:06.783185 15607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:06.783469 15600 server_base.cc:1034] running on GCE node
I20260811 05:11:06.783950 15600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:06.784550 15600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:06.785718 15600 hybrid_clock.cc:648] HybridClock initialized: now 1786425066785695 us; error 24 us; skew 500 ppm
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:06.788345 15600 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:11:06.789248 15600 webserver.cc:460] Webserver started at http://127.30.88.195:37245/ using document root <none> and password file <none>
I20260811 05:11:06.789793 15600 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:06.789901 15600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:06.793517 15600 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 05:11:06.795531 15617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.796679 15600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:11:06.796803 15600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f044e811a7a841b4b402e9bedc5c2428"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "231c3040c93dc68aef8d11a6ebae2f00"
server_key_iv: "da8cd9068a420628bea87acacbfa4bfb"
server_key_version: "encryptionkey@0"
I20260811 05:11:06.797286 15600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:06.814621 15600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:06.817572 15600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:06.817744 15600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:06.818328 15600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:06.819175 15600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:06.819245 15600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.819311 15600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:06.819365 15600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:06.828225 15600 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:46757
I20260811 05:11:06.828348 15741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:46757 every 8 connection(s)
I20260811 05:11:06.829267 15600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:11:06.835435 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15600
Aug 11 05:11:06 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:06.842772 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.830957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:33697 (local address 127.30.88.254:39199)
0811 05:11:06.831090 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.831094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.831885 (+ 791us) server_negotiation.cc:407] Connection header received
0811 05:11:06.833118 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.833122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.833179 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.833276 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:06.834296 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.834778 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.835840 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.836120 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.836216 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.838897 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:06.838900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:06.838903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:06.838929 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:06.840732 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.841267 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.841270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.841271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.841347 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.841731 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.841734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.841735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.841952 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:06.842067 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.842479 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.842595 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:11:06.843534 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.831317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:33697)
0811 05:11:06.831754 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.831789 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.832911 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.833436 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.833441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.833736 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:06.834142 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.834154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.834913 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.834917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.835722 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.835731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.836333 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.836340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.836554 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.837054 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:06.837079 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:06.838705 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:06.840841 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.840846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.840858 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.841170 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.841438 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:06.841441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:06.841444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:06.841619 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:06.842083 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:06.842088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:06.842266 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.842398 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.842594 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260811 05:11:06.844601 15742 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39199
I20260811 05:11:06.844707 15746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.836051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.30.88.254:39199)
0811 05:11:06.836260 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.836264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.836381 (+ 117us) server_negotiation.cc:407] Connection header received
0811 05:11:06.836510 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.836512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.836541 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.836612 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:06.837525 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.838180 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.839063 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.839449 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.839584 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.841059 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:06.841071 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:06.841075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:06.841099 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:06.842615 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.843230 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.843233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.843234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.843304 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:06.843576 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:06.843577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:06.843578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:06.843781 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:06.843932 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.844435 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.844543 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":65,"threads_started":1}
I20260811 05:11:06.844818 15742 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:06.845290 15742 heartbeater.cc:511] Master 127.30.88.254:39199 requested a full tablet report, sending...
I20260811 05:11:06.846275 15220 ts_manager.cc:194] Registered new tserver with Master: f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:06.846841 15220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:33697
I20260811 05:11:06.857800 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:11:06.861465 15746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.858349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.30.88.254:39199)
0811 05:11:06.858481 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.858484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.858582 (+ 98us) server_negotiation.cc:407] Connection header received
0811 05:11:06.858671 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.858673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.858744 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.858834 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:06.859080 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.859520 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.860311 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.860498 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.860566 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.860847 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:06.861181 (+ 334us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:11:06.861257 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.861311 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.861351 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:11:06.861992 15220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:11:06.865051 15220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:11:06.872337 15360 tablet_service.cc:1467] Processing CreateTablet for tablet b51d4a78aa2a44678aa2b93052031f5c (DEFAULT_TABLE table=test-table [id=7fd02d06c88f4742804e9a3d142cfcbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:06.872997 15360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b51d4a78aa2a44678aa2b93052031f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:06.875795 15313 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Bootstrap starting.
I20260811 05:11:06.876727 15313 tablet_bootstrap.cc:654] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:06.876736 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.871267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43485 (local address 127.0.0.1:42860)
0811 05:11:06.871375 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.871390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.871472 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.872341 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.872344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.872368 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:06.872463 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.872469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.873782 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.873785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.875071 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.875082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.876260 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.876264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.876360 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.876402 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.876480 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.876559 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":67}
I20260811 05:11:06.877020 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.871880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:46757 (local address 127.0.0.1:55446)
0811 05:11:06.872141 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:06.872154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:06.872266 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:06.872999 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:06.873002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:06.873020 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:06.873190 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.873198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.874083 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.874089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.875273 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:06.875281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.876458 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.876464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:06.876551 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.876584 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:11:06.876652 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:11:06.876728 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":57,"thread_start_us":61,"threads_started":1}
I20260811 05:11:06.877034 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.871463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.30.88.194:43485)
0811 05:11:06.871746 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.871759 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.871782 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:11:06.871857 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.871862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.872090 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.872354 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:06.872609 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.873641 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.875198 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.876089 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.876243 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.876395 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.876525 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.876625 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260811 05:11:06.877599 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:06.872001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.30.88.195:46757)
0811 05:11:06.872658 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:06.872667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:06.872692 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:11:06.872744 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:06.872747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:06.872864 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:06.873012 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:06.873320 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.873955 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.875386 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:06.876214 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:06.876298 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:06.876394 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:06.877382 (+ 988us) server_negotiation.cc:299] Negotiation successful
0811 05:11:06.877469 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":99,"threads_started":1}
I20260811 05:11:06.878391 15313 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: No bootstrap required, opened a new log
I20260811 05:11:06.878484 15313 ts_tablet_manager.cc:1397] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:11:06.878957 15313 raft_consensus.cc:348] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.879086 15313 raft_consensus.cc:374] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:06.879123 15313 raft_consensus.cc:729] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c190330708f40088a613734f31fc185, State: Initialized, Role: FOLLOWER
I20260811 05:11:06.879251 15313 consensus_queue.cc:260] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.879629 15313 ts_tablet_manager.cc:1428] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:11:06.880053 15517 tablet_service.cc:1467] Processing CreateTablet for tablet b51d4a78aa2a44678aa2b93052031f5c (DEFAULT_TABLE table=test-table [id=7fd02d06c88f4742804e9a3d142cfcbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:06.880321 15668 tablet_service.cc:1467] Processing CreateTablet for tablet b51d4a78aa2a44678aa2b93052031f5c (DEFAULT_TABLE table=test-table [id=7fd02d06c88f4742804e9a3d142cfcbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:06.881256 15517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b51d4a78aa2a44678aa2b93052031f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:06.881279 15668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b51d4a78aa2a44678aa2b93052031f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:06.887879 15756 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Bootstrap starting.
I20260811 05:11:06.887879 15757 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Bootstrap starting.
I20260811 05:11:06.890058 15756 tablet_bootstrap.cc:654] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:06.890062 15757 tablet_bootstrap.cc:654] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:06.890777 15756 log.cc:824] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:06.890777 15757 log.cc:824] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:06.892645 15757 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: No bootstrap required, opened a new log
I20260811 05:11:06.892832 15757 ts_tablet_manager.cc:1397] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 05:11:06.892874 15756 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: No bootstrap required, opened a new log
I20260811 05:11:06.893054 15756 ts_tablet_manager.cc:1397] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:11:06.895604 15757 raft_consensus.cc:348] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.895646 15756 raft_consensus.cc:348] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.895825 15757 raft_consensus.cc:374] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:06.895818 15756 raft_consensus.cc:374] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:06.895898 15756 raft_consensus.cc:729] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7678cdbca2e84ec5808bc8ce28428561, State: Initialized, Role: FOLLOWER
I20260811 05:11:06.895900 15757 raft_consensus.cc:729] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f044e811a7a841b4b402e9bedc5c2428, State: Initialized, Role: FOLLOWER
I20260811 05:11:06.896323 15756 consensus_queue.cc:260] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.896322 15757 consensus_queue.cc:260] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:06.897254 15757 ts_tablet_manager.cc:1428] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260811 05:11:06.897265 15756 ts_tablet_manager.cc:1428] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 05:11:06.897286 15596 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
I20260811 05:11:06.898264 15742 heartbeater.cc:503] Master 127.30.88.254:39199 was elected leader, sending a full tablet report...
W20260811 05:11:06.899935 15597 tablet.cc:2369] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:11:06.973484 15435 tablet.cc:2369] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:11:07.080228 15743 tablet.cc:2369] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:07.195663 15441 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:07.195964 15441 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:07.197073 15441 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:07.202186 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.197913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:35249 (local address 127.30.88.195:46757)
0811 05:11:07.198041 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:07.198045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:07.198301 (+ 256us) server_negotiation.cc:407] Connection header received
0811 05:11:07.198418 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:07.198420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:07.198480 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:07.198558 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:07.198892 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.199534 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.200855 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.201425 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.201533 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:07.201573 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:07.201960 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 05:11:07.202048 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:11:07.202234 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.197761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:46757 (local address 127.30.88.193:35249)
0811 05:11:07.198208 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.198221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:07.198312 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:07.198612 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:07.198615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:07.198652 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:07.198736 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:07.198741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.199640 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.199643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:07.200745 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:07.200755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.201528 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.201531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:07.201753 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:07.201845 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 05:11:07.201934 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:11:07.202049 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":97,"thread_start_us":79,"threads_started":1}
I20260811 05:11:07.202265 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.197494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:34753 (local address 127.30.88.194:43485)
0811 05:11:07.197627 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:07.197631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:07.197747 (+ 116us) server_negotiation.cc:407] Connection header received
0811 05:11:07.197949 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:07.197953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:07.198013 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:07.198102 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:07.198431 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.199251 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.200750 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.201396 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.201524 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:07.201573 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:07.202059 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 05:11:07.202155 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:11:07.202276 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.197367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43485 (local address 127.30.88.193:34753)
0811 05:11:07.197644 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.197661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:07.197788 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:07.198147 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:07.198150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:07.198171 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:07.198313 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:07.198320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.199368 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.199370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:07.200616 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:07.200628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.201528 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.201531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:07.201757 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:07.201845 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 05:11:07.201942 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 05:11:07.202091 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":114,"thread_start_us":112,"threads_started":1}
I20260811 05:11:07.202968 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b51d4a78aa2a44678aa2b93052031f5c" candidate_uuid: "5c190330708f40088a613734f31fc185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f044e811a7a841b4b402e9bedc5c2428" is_pre_election: true
I20260811 05:11:07.202975 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b51d4a78aa2a44678aa2b93052031f5c" candidate_uuid: "5c190330708f40088a613734f31fc185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7678cdbca2e84ec5808bc8ce28428561" is_pre_election: true
I20260811 05:11:07.203321 15539 raft_consensus.cc:2454] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c190330708f40088a613734f31fc185 in term 0.
I20260811 05:11:07.203321 15691 raft_consensus.cc:2454] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c190330708f40088a613734f31fc185 in term 0.
I20260811 05:11:07.203758 15308 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c190330708f40088a613734f31fc185, f044e811a7a841b4b402e9bedc5c2428; no voters:
I20260811 05:11:07.203997 15441 raft_consensus.cc:2781] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:11:07.204077 15441 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:11:07.204106 15441 raft_consensus.cc:3037] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:07.205108 15441 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:07.205439 15441 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 1 election: Requested vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
I20260811 05:11:07.205888 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b51d4a78aa2a44678aa2b93052031f5c" candidate_uuid: "5c190330708f40088a613734f31fc185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f044e811a7a841b4b402e9bedc5c2428"
I20260811 05:11:07.205883 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b51d4a78aa2a44678aa2b93052031f5c" candidate_uuid: "5c190330708f40088a613734f31fc185" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7678cdbca2e84ec5808bc8ce28428561"
I20260811 05:11:07.206012 15691 raft_consensus.cc:3037] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:07.206012 15539 raft_consensus.cc:3037] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:07.207304 15691 raft_consensus.cc:2454] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c190330708f40088a613734f31fc185 in term 1.
I20260811 05:11:07.207304 15539 raft_consensus.cc:2454] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c190330708f40088a613734f31fc185 in term 1.
I20260811 05:11:07.207556 15308 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c190330708f40088a613734f31fc185, f044e811a7a841b4b402e9bedc5c2428; no voters:
I20260811 05:11:07.207718 15441 raft_consensus.cc:2781] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:07.207937 15441 raft_consensus.cc:686] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 LEADER]: Becoming Leader. State: Replica: 5c190330708f40088a613734f31fc185, State: Running, Role: LEADER
I20260811 05:11:07.208209 15441 consensus_queue.cc:237] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:07.211099 15221 catalog_manager.cc:5495] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c190330708f40088a613734f31fc185 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "5c190330708f40088a613734f31fc185" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260811 05:11:07.240424 15769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.237209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.30.88.193:40731)
0811 05:11:07.237462 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:07.237468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:07.237479 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:11:07.237537 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:07.237539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:07.237592 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:07.237682 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:07.237874 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.238309 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.239073 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:07.239287 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:07.239363 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:07.239589 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:07.240017 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:11:07.240106 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:07.240209 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 05:11:07.240249 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":121,"threads_started":1}
I20260811 05:11:07.243889 15539 raft_consensus.cc:1261] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Refusing update from remote peer 5c190330708f40088a613734f31fc185: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:07.243889 15691 raft_consensus.cc:1261] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Refusing update from remote peer 5c190330708f40088a613734f31fc185: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:07.244597 15766 consensus_queue.cc:1035] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:07.244786 15441 consensus_queue.cc:1035] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:07.251097 15770 mvcc.cc:205] Tried to move back new op lower bound from 7317197075425181696 to 7317197075287404544. Current Snapshot: MvccSnapshot[applied={T|T < 7317197075425181696}]
I20260811 05:11:07.258585 15221 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:07.258724 15221 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:07.260874 15221 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Sending DeleteTablet for 3 replicas of tablet b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:07.261442 15517 tablet_service.cc:1514] Processing DeleteTablet for tablet b51d4a78aa2a44678aa2b93052031f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:07 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:42860
I20260811 05:11:07.261538 15668 tablet_service.cc:1514] Processing DeleteTablet for tablet b51d4a78aa2a44678aa2b93052031f5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:07 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:55446
I20260811 05:11:07.261600 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 15268
I20260811 05:11:07.265450 15782 tablet_replica.cc:330] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: stopping tablet replica
I20260811 05:11:07.265729 15782 raft_consensus.cc:2229] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:07.266000 15782 raft_consensus.cc:2258] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:07.266201 15781 tablet_replica.cc:330] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: stopping tablet replica
I20260811 05:11:07.266484 15781 raft_consensus.cc:2229] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:07.266759 15781 raft_consensus.cc:2258] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:07.267391 15782 ts_tablet_manager.cc:1905] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:07.268173 15781 ts_tablet_manager.cc:1905] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:07.269658 15782 ts_tablet_manager.cc:1918] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:07.269778 15782 log.cc:1196] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:07.270144 15782 ts_tablet_manager.cc:1939] T b51d4a78aa2a44678aa2b93052031f5c P f044e811a7a841b4b402e9bedc5c2428: Deleting consensus metadata
I20260811 05:11:07.271232 15207 catalog_manager.cc:4841] TS f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): tablet b51d4a78aa2a44678aa2b93052031f5c (table test-table [id=7fd02d06c88f4742804e9a3d142cfcbb]) successfully deleted
I20260811 05:11:07.271514 15781 ts_tablet_manager.cc:1918] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:07.271608 15781 log.cc:1196] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:07.271925 15781 ts_tablet_manager.cc:1939] T b51d4a78aa2a44678aa2b93052031f5c P 7678cdbca2e84ec5808bc8ce28428561: Deleting consensus metadata
W20260811 05:11:07.273226 15205 connection.cc:535] client connection to 127.30.88.193:40731 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:11:07.273272 15207 catalog_manager.cc:4841] TS 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): tablet b51d4a78aa2a44678aa2b93052031f5c (table test-table [id=7fd02d06c88f4742804e9a3d142cfcbb]) successfully deleted
W20260811 05:11:07.273386 15205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:11:07.273609 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 15439
I20260811 05:11:07.276686 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.276336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40731 (local address 127.0.0.1:44130)
0811 05:11:07.276513 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.276590 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 05:11:07.276868 15205 catalog_manager.cc:4584] TS 5c190330708f40088a613734f31fc185 (127.30.88.193:40731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b51d4a78aa2a44678aa2b93052031f5c: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40731: connect: Connection refused (error 111)
I20260811 05:11:07.281894 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 15600
I20260811 05:11:07.289440 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 15186
I20260811 05:11:07.297503 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--webserver_interface=127.30.88.254
--webserver_port=39621
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:39199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:07.406222 15784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:07.406469 15784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:07.406539 15784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:07.410195 15784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:07.410276 15784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:07.410316 15784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:07.410374 15784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:07.410430 15784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:07.415369 15784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:39199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=39621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:07.416250 15784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:07.417161 15784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:07.430183 15791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:07.430235 15793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:07.430197 15790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:07.430667 15784 server_base.cc:1034] running on GCE node
I20260811 05:11:07.431260 15784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:07.432226 15784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:07.433422 15784 hybrid_clock.cc:648] HybridClock initialized: now 1786425067433379 us; error 44 us; skew 500 ppm
Aug 11 05:11:07 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:07.440522 15784 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:07.441349 15784 webserver.cc:460] Webserver started at http://127.30.88.254:39621/ using document root <none> and password file <none>
I20260811 05:11:07.441893 15784 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:07.441979 15784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:07.445518 15784 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:07.447433 15801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:07.448593 15784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:07.448740 15784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1d639b3fce16408e8ce797dd5d7a54a5"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "3e2f05f6444f473af92cee0562a504bf"
server_key_iv: "4cdd13985ad458778469753d9f354dec"
server_key_version: "encryptionkey@0"
I20260811 05:11:07.449262 15784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:07.464954 15784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:07.467414 15784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:07.467638 15784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:07.473985 15784 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:39199
I20260811 05:11:07.474231 15858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:39199 every 8 connection(s)
I20260811 05:11:07.475045 15784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:07.479080 15859 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:11:07.480628 15859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap starting.
I20260811 05:11:07.483619 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15784
I20260811 05:11:07.484051 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:40731
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=43785
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:07.497493 15859 log.cc:824] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:07.508365 15859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:07.508718 15859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5: Bootstrap complete.
I20260811 05:11:07.512470 15859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:07.512815 15859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Initialized, Role: FOLLOWER
I20260811 05:11:07.513250 15859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:07.513434 15859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:07.513515 15859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:07.513628 15859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:11:07.515443 15859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:07.515764 15859 leader_election.cc:302] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d639b3fce16408e8ce797dd5d7a54a5; no voters:
I20260811 05:11:07.516093 15859 leader_election.cc:288] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:11:07.516170 15864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:11:07.516398 15864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [term 3 LEADER]: Becoming Leader. State: Replica: 1d639b3fce16408e8ce797dd5d7a54a5, State: Running, Role: LEADER
I20260811 05:11:07.516682 15864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } }
I20260811 05:11:07.517300 15859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:07.517553 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:07.517678 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:07.518211 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d639b3fce16408e8ce797dd5d7a54a5. Latest consensus state: current_term: 3 leader_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d639b3fce16408e8ce797dd5d7a54a5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39199 } } }
I20260811 05:11:07.518316 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d639b3fce16408e8ce797dd5d7a54a5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:07.518388 15870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:07.523301 15870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:07.523885 15870 catalog_manager.cc:1259] Loaded cluster ID: c24c17a8003846958ecf3125274c361b
I20260811 05:11:07.523942 15870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 05:11:07.524941 15870 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae2b3c69ad google::LogMessage::Fail() at ??:0
@ 0x7fae2b3c988a google::LogMessage::SendToLog() at ??:0
@ 0x7fae2b3c64de google::LogMessage::Flush() at ??:0
@ 0x7fae2b3c7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae35d05fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fae35d06d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae35d03579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fae35d47640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fae2d6183d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae2d618c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fae2d61a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fae2d609512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae2ec4d6db start_thread at ??:0
@ 0x7fae29ec461f clone at ??:0
W20260811 05:11:07.608640 15861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:07.608919 15861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:07.609026 15861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:07.613427 15861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:07.613493 15861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:07.613581 15861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:11:07.621294 15861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:40731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=43785
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:07.622260 15861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:07.623334 15861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:07.632438 15906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:07.632432 15909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:07.632778 15861 server_base.cc:1034] running on GCE node
W20260811 05:11:07.632586 15907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:07.633172 15861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:07.633769 15861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:07.634925 15861 hybrid_clock.cc:648] HybridClock initialized: now 1786425067634908 us; error 31 us; skew 500 ppm
Aug 11 05:11:07 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:07.637528 15861 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:11:07.638283 15861 webserver.cc:460] Webserver started at http://127.30.88.193:43785/ using document root <none> and password file <none>
I20260811 05:11:07.638770 15861 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:07.638916 15861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:07.642817 15861 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:11:07.644894 15917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:07.646107 15861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:07.646231 15861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "5c190330708f40088a613734f31fc185"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "24949c5154634a70173caa3d68375c19"
server_key_iv: "5cd01421b707689566b01695e7469e95"
server_key_version: "encryptionkey@0"
I20260811 05:11:07.646776 15861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:07.660221 15861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:07.663157 15861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:07.663326 15861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:07.663823 15861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:07.664804 15925 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:11:07.669899 15861 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:11:07.669987 15861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 05:11:07.670069 15861 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:11:07.671837 15861 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:11:07.671902 15861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:07.672184 15925 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Bootstrap starting.
I20260811 05:11:07.682585 15861 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:40731
I20260811 05:11:07.682663 16041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:40731 every 8 connection(s)
I20260811 05:11:07.683531 15861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:11:07.692063 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 15861
I20260811 05:11:07.692492 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:43485
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=45691
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:07.693449 15925 log.cc:824] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:07.702414 15925 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:07.702833 15925 tablet_bootstrap.cc:492] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Bootstrap complete.
I20260811 05:11:07.704197 15925 ts_tablet_manager.cc:1397] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Time spent bootstrapping tablet: real 0.032s user 0.022s sys 0.008s
I20260811 05:11:07.707173 15925 raft_consensus.cc:348] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:07.707684 15925 raft_consensus.cc:729] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c190330708f40088a613734f31fc185, State: Initialized, Role: FOLLOWER
I20260811 05:11:07.708187 15925 consensus_queue.cc:260] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:07.709192 15925 ts_tablet_manager.cc:1428] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 05:11:07.737479 16044 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:11:07.685301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:54249)
0811 05:11:07.685750 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.685798 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:07.686533 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:07.736663 (+ 50130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
W20260811 05:11:07.738075 16042 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:39199 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:39199: Client connection negotiation failed: client connection to 127.30.88.254:39199: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:11:07.738921 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.738551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:43671)
0811 05:11:07.738710 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.738795 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 05:11:07.739742 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.739463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:47409)
0811 05:11:07.739601 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.739667 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 05:11:07.739933 16042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:11:07.805019 16046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:07.805228 16046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:07.805280 16046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:07.808769 16046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:07.808811 16046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:07.808863 16046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:11:07.813369 16046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:43485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=45691
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:07.814251 16046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:07.815174 16046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:07.824168 16056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:07.824178 16046 server_base.cc:1034] running on GCE node
W20260811 05:11:07.824122 16054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:07.824126 16053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:07.824743 16046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:07.825244 16046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:07.826460 16046 hybrid_clock.cc:648] HybridClock initialized: now 1786425067826441 us; error 29 us; skew 500 ppm
Aug 11 05:11:07 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:07.829175 16046 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:11:07.829970 16046 webserver.cc:460] Webserver started at http://127.30.88.194:45691/ using document root <none> and password file <none>
I20260811 05:11:07.830498 16046 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:07.830559 16046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:07.834010 16046 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:07.835829 16063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:07.836990 16046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:07.837149 16046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "7678cdbca2e84ec5808bc8ce28428561"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "9a98500da5d5cee014741b304f74e7cb"
server_key_iv: "cc4a0028853fc57ff035e82397b526b2"
server_key_version: "encryptionkey@0"
I20260811 05:11:07.837657 16046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:07.864871 16046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:07.867333 16046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:07.867529 16046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:07.868058 16046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:07.868798 16046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:07.868863 16046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:07.868942 16046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:07.868997 16046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:07.877370 16046 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:43485
I20260811 05:11:07.877401 16192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:43485 every 8 connection(s)
I20260811 05:11:07.878330 16046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:11:07.881738 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.880366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:57691)
0811 05:11:07.880765 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.881139 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
W20260811 05:11:07.882119 16193 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:39199 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:39199: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
I20260811 05:11:07.882635 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.882452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:38351)
0811 05:11:07.882544 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.882586 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:11:07.883196 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:07.883031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:58723)
0811 05:11:07.883116 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:07.883154 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 05:11:07.883347 16193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:11:07.884618 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16046
I20260811 05:11:07.884958 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:46757
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:39199
--webserver_port=37245
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:07.987112 16196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:07.987313 16196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:07.987350 16196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:07.991314 16196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:07.991359 16196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:07.991457 16196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:11:07.996563 16196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:46757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=37245
--tserver_master_addrs=127.30.88.254:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:07.997697 16196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:07.998651 16196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:08.007350 16203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:08.007366 16205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:08.007366 16202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:08.007565 16196 server_base.cc:1034] running on GCE node
I20260811 05:11:08.008023 16196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:08.008572 16196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:08.009753 16196 hybrid_clock.cc:648] HybridClock initialized: now 1786425068009744 us; error 26 us; skew 500 ppm
Aug 11 05:11:08 dist-test-slave-7c35 krb5kdc[14582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425068, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:08.012497 16196 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:11:08.013314 16196 webserver.cc:460] Webserver started at http://127.30.88.195:37245/ using document root <none> and password file <none>
I20260811 05:11:08.013821 16196 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:08.013909 16196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:08.017242 16196 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:08.019258 16214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:08.020387 16196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:08.020558 16196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f044e811a7a841b4b402e9bedc5c2428"
format_stamp: "Formatted at 2026-08-11 05:11:05 on dist-test-slave-7c35"
server_key: "231c3040c93dc68aef8d11a6ebae2f00"
server_key_iv: "da8cd9068a420628bea87acacbfa4bfb"
server_key_version: "encryptionkey@0"
I20260811 05:11:08.021248 16196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:08.033638 16196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:08.036070 16196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:08.036247 16196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:08.036756 16196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:08.037448 16196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:08.037492 16196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:08.037547 16196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:08.037580 16196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:08.046026 16196 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:46757
I20260811 05:11:08.046046 16338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:46757 every 8 connection(s)
I20260811 05:11:08.046878 16196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:11:08.050028 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:08.048604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:55625)
0811 05:11:08.049226 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:08.049329 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":106,"threads_started":1}
W20260811 05:11:08.050449 16339 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:39199 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:39199: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
I20260811 05:11:08.051067 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:08.050778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:53627)
0811 05:11:08.050968 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:08.051010 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 05:11:08.051503 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16196
I20260811 05:11:08.051684 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:08.051494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:56281)
0811 05:11:08.051585 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:08.051628 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 05:11:08.051858 16339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:11:08.740911 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:08.740542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:57929)
0811 05:11:08.740760 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:08.740808 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260811 05:11:08.884268 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:08.883910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:35597)
0811 05:11:08.884126 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:08.884175 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260811 05:11:09.052627 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.052286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:32783)
0811 05:11:09.052492 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:09.052537 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260811 05:11:09.322597 16347 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:09.322814 16347 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:09.323895 16347 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
Aug 11 05:11:09 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.194@KRBTEST.COM
Aug 11 05:11:09 dist-test-slave-7c35 krb5kdc[14582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.195@KRBTEST.COM
I20260811 05:11:09.338161 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.324127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43485 (local address 127.30.88.193:43441)
0811 05:11:09.324469 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:09.324483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:09.324614 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:09.326151 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:09.326157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:09.326639 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:09.327173 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:09.327188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.328386 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.328389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:09.329612 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:09.329624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.330538 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.330540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:09.330679 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:09.331268 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:09.331288 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:09.332955 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:09.335434 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:09.335439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:09.335450 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:09.335959 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:09.336401 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:09.336407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:09.336636 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:09.337464 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:09.337470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:09.337665 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 05:11:09.337836 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 05:11:09.338032 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":2,"spinlock_wait_cycles":6784,"thread_start_us":68,"threads_started":1}
I20260811 05:11:09.338344 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.324254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:43441 (local address 127.30.88.194:43485)
0811 05:11:09.324596 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:09.324624 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:09.324671 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:11:09.325472 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:09.325485 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:09.325804 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:09.326040 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:09.327345 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.328237 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.329735 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.330402 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.330580 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:09.333182 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:09.333187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:09.333203 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:09.333235 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:09.335283 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336111 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:09.336117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:09.336122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:09.336276 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336747 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:09.336750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:09.336752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:09.337224 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:09.337356 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:09.337935 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 05:11:09.338137 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260811 05:11:09.339003 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.324528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:46757 (local address 127.30.88.193:43993)
0811 05:11:09.324810 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:09.324819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:09.324884 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:09.326518 (+ 1634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:09.326521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:09.326829 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:09.327172 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:09.327187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.329916 (+ 2729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.329919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:09.330720 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:09.330739 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.331632 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.331635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:09.331712 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:09.332169 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:09.332180 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:09.333361 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:09.335600 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:09.335603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:09.335605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:09.335966 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:09.336401 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:09.336407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:09.336636 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:09.338357 (+ 1721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:09.338360 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:09.338492 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 05:11:09.338659 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 05:11:09.338798 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":24,"spinlock_wait_cycles":6784,"thread_start_us":52,"threads_started":1}
I20260811 05:11:09.339373 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.324783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:43993 (local address 127.30.88.195:46757)
0811 05:11:09.325101 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:09.325119 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:09.325149 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:11:09.325770 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:09.325776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:09.326100 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:09.326356 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:09.328433 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.329805 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.330869 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:09.331505 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:09.331680 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:09.333539 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:09.333547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:09.333562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:09.333587 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:09.335497 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336111 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:09.336117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:09.336122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:09.336276 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:09.336747 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:09.336750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:09.336752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:09.337224 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:09.337356 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:09.338797 (+ 1441us) server_negotiation.cc:299] Negotiation successful
0811 05:11:09.339236 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":119,"threads_started":1}
W20260811 05:11:09.341013 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
W20260811 05:11:09.341398 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:09.341473 15920 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c190330708f40088a613734f31fc185; no voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428
I20260811 05:11:09.341768 16347 raft_consensus.cc:2726] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:11:09.742020 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.741722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:36861)
0811 05:11:09.741871 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:09.741934 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:11:09.885272 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:09.884893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:56455)
0811 05:11:09.885144 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:09.885192 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260811 05:11:10.053580 16355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:10.053232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:52429)
0811 05:11:10.053436 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:10.053491 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":63,"threads_started":1}
I20260811 05:11:10.743063 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:10.742636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:40743)
0811 05:11:10.742907 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:10.742954 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
I20260811 05:11:10.886381 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:10.886021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:42377)
0811 05:11:10.886238 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:10.886286 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260811 05:11:10.997001 16358 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:10.997130 16358 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:10.997562 16358 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
W20260811 05:11:10.998426 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
W20260811 05:11:10.998610 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:10.998664 15920 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c190330708f40088a613734f31fc185; no voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428
I20260811 05:11:10.998824 16358 raft_consensus.cc:2726] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:11:11.054508 16359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:11.054194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:50013)
0811 05:11:11.054390 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:11.054435 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":64,"threads_started":1}
I20260811 05:11:11.744215 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:11.743798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:41901)
0811 05:11:11.744069 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:11.744124 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":98,"threads_started":1}
I20260811 05:11:11.887502 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:11.887133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:50857)
0811 05:11:11.887343 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:11.887389 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260811 05:11:12.055485 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:12.055140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:40763)
0811 05:11:12.055368 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:12.055413 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260811 05:11:12.745651 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:12.745149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:40573)
0811 05:11:12.745491 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:12.745544 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":69,"threads_started":1}
I20260811 05:11:12.888517 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:12.888170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:55257)
0811 05:11:12.888396 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:12.888441 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260811 05:11:13.056557 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:13.056102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:37955)
0811 05:11:13.056395 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:13.056458 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":123,"threads_started":1}
I20260811 05:11:13.722328 16367 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:13.722460 16367 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:13.722916 16367 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
W20260811 05:11:13.723860 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
W20260811 05:11:13.723980 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:13.724062 15920 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c190330708f40088a613734f31fc185; no voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428
I20260811 05:11:13.724206 16367 raft_consensus.cc:2726] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:11:13.746815 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:13.746437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:39161)
0811 05:11:13.746684 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:13.746731 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260811 05:11:13.889638 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:13.889234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:47177)
0811 05:11:13.889456 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:13.889511 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260811 05:11:14.057677 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:14.057310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:34717)
0811 05:11:14.057537 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:14.057597 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260811 05:11:14.747946 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:14.747562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:50569)
0811 05:11:14.747799 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:14.747850 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260811 05:11:14.890918 16373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:14.890368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:48611)
0811 05:11:14.890695 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:14.890751 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":91,"threads_started":1}
I20260811 05:11:15.058810 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:15.058381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:54029)
0811 05:11:15.058670 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:15.058722 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":81,"threads_started":1}
I20260811 05:11:15.749619 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:15.748970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:37863)
0811 05:11:15.749360 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:15.749447 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":112,"threads_started":1}
I20260811 05:11:15.892407 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:15.891824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:56907)
0811 05:11:15.892165 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:15.892248 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":85,"threads_started":1}
I20260811 05:11:16.061172 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:16.060096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:41927)
0811 05:11:16.060673 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:16.060867 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":186,"threads_started":1}
I20260811 05:11:16.751408 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:16.750745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:50351)
0811 05:11:16.751155 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:16.751264 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":143,"threads_started":1}
I20260811 05:11:16.894083 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:16.893389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:45679)
0811 05:11:16.893835 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:16.893933 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":100,"threads_started":1}
I20260811 05:11:17.063447 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:17.062603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:37605)
0811 05:11:17.063109 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:17.063225 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":182,"threads_started":1}
I20260811 05:11:17.566759 16381 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:17.567032 16381 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:17.567739 16381 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
W20260811 05:11:17.568988 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
W20260811 05:11:17.569240 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:17.569331 15920 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c190330708f40088a613734f31fc185; no voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428
I20260811 05:11:17.569622 16381 raft_consensus.cc:2726] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:11:17.753825 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:17.752959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:45959)
0811 05:11:17.753448 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:17.753603 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":143,"threads_started":1}
I20260811 05:11:17.895823 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:17.895213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:51363)
0811 05:11:17.895603 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:17.895686 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
I20260811 05:11:18.064744 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:18.064354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:46837)
0811 05:11:18.064607 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:18.064657 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260811 05:11:18.755254 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:18.754765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:52261)
0811 05:11:18.755059 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:18.755125 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":135,"threads_started":1}
I20260811 05:11:18.897135 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:18.896704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:57243)
0811 05:11:18.896986 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:18.897042 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260811 05:11:19.065991 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:19.065583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:60501)
0811 05:11:19.065844 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:19.065899 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260811 05:11:19.756428 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:19.756025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:48443)
0811 05:11:19.756276 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:19.756328 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260811 05:11:19.898408 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:19.898039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:36499)
0811 05:11:19.898269 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:19.898321 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260811 05:11:20.067158 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:20.066686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:59367)
0811 05:11:20.067015 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:20.067069 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":83,"threads_started":1}
I20260811 05:11:20.101372 16391 raft_consensus.cc:482] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:20.101511 16391 raft_consensus.cc:504] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7678cdbca2e84ec5808bc8ce28428561" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43485 } } peers { permanent_uuid: "f044e811a7a841b4b402e9bedc5c2428" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 46757 } } peers { permanent_uuid: "5c190330708f40088a613734f31fc185" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40731 } }
I20260811 05:11:20.101948 16391 leader_election.cc:288] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485), f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757)
W20260811 05:11:20.102901 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f044e811a7a841b4b402e9bedc5c2428 (127.30.88.195:46757): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
W20260811 05:11:20.103080 15920 leader_election.cc:341] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7678cdbca2e84ec5808bc8ce28428561 (127.30.88.194:43485): Not found: Tablet not found: b51d4a78aa2a44678aa2b93052031f5c
I20260811 05:11:20.103139 15920 leader_election.cc:302] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c190330708f40088a613734f31fc185; no voters: 7678cdbca2e84ec5808bc8ce28428561, f044e811a7a841b4b402e9bedc5c2428
I20260811 05:11:20.103297 16391 raft_consensus.cc:2726] T b51d4a78aa2a44678aa2b93052031f5c P 5c190330708f40088a613734f31fc185 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:11:20.757678 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:20.757271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:49343)
0811 05:11:20.757530 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:20.757582 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260811 05:11:20.899595 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:20.899215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:58407)
0811 05:11:20.899450 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:20.899498 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260811 05:11:21.068186 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:21.067822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:38195)
0811 05:11:21.068045 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:21.068103 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":76,"threads_started":1}
I20260811 05:11:21.758816 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:21.758393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:34065)
0811 05:11:21.758651 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:21.758709 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260811 05:11:21.900836 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:21.900441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:45555)
0811 05:11:21.900686 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:21.900741 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260811 05:11:22.069308 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:22.068845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:34737)
0811 05:11:22.069165 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:22.069219 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":70,"threads_started":1}
I20260811 05:11:22.760154 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:22.759762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.193:33999)
0811 05:11:22.760009 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:22.760058 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260811 05:11:22.902074 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:22.901691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.194:45483)
0811 05:11:22.901926 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:22.901974 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
I20260811 05:11:23.054709 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 15861
I20260811 05:11:23.064474 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16046
I20260811 05:11:23.070580 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:23.070191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39199 (local address 127.30.88.195:45029)
0811 05:11:23.070430 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:23.070490 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:39199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260811 05:11:23.072981 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16196
2026-08-11T05:11:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19318 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:11:23 dist-test-slave-7c35 krb5kdc[16404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:11:23 dist-test-slave-7c35 krb5kdc[16404](info): set up 2 sockets
Aug 11 05:11:23 dist-test-slave-7c35 krb5kdc[16404](info): commencing operation
krb5kdc: starting...
W20260811 05:11:24.155973 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:11:24.279829 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:24.388340 16418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:24.388597 16418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:24.388636 16418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:24.392563 16418 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:11:24.392607 16418 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:11:24.392622 16418 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:11:24.392635 16418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:24.392652 16418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:24.392669 16418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:24.392684 16418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:24.392702 16418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:24.397663 16418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:24.398562 16418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:24.399561 16418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:24.407895 16424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:24.407998 16418 server_base.cc:1034] running on GCE node
W20260811 05:11:24.407895 16423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:24.407965 16426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:24.408679 16418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:24.409576 16418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:24.410763 16418 hybrid_clock.cc:648] HybridClock initialized: now 1786425084410740 us; error 41 us; skew 500 ppm
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:24.413393 16418 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:24.414192 16418 webserver.cc:460] Webserver started at http://127.30.88.254:35015/ using document root <none> and password file <none>
I20260811 05:11:24.414702 16418 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:24.414757 16418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:24.414986 16418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:24.416913 16418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "2f43a28146084c6a8722ce404097e29c"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "84b050e19b041a95d80b31a936127954"
server_key_iv: "2d9e95e24f503ae9d4eb8c5a06b8fa6b"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.417416 16418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "2f43a28146084c6a8722ce404097e29c"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "84b050e19b041a95d80b31a936127954"
server_key_iv: "2d9e95e24f503ae9d4eb8c5a06b8fa6b"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.420789 16418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:11:24.423120 16433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.424280 16418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:11:24.424373 16418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "2f43a28146084c6a8722ce404097e29c"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "84b050e19b041a95d80b31a936127954"
server_key_iv: "2d9e95e24f503ae9d4eb8c5a06b8fa6b"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.424455 16418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:24.441542 16418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:24.444221 16418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:24.444362 16418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:24.450803 16418 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35195
I20260811 05:11:24.451097 16492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35195 every 8 connection(s)
I20260811 05:11:24.451758 16418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:24.454245 16493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:24.455821 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16418
I20260811 05:11:24.455911 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:11:24.456166 31075 external_mini_cluster.cc:1428] Setting key ae9a7acbb12e30bff2211b831c38537e
I20260811 05:11:24.460038 16493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap starting.
I20260811 05:11:24.462486 16493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:24.463433 16493 log.cc:824] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:24.465327 16493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: No bootstrap required, opened a new log
I20260811 05:11:24.469542 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:24.457303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.30.88.254:35195)
0811 05:11:24.457651 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:24.457671 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:24.457701 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:11:24.458242 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:24.458255 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:24.458518 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:24.458760 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:24.459671 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.460706 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.461926 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.462350 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.462455 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:24.465080 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:24.465086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:24.465099 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:24.465128 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:24.466888 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.467250 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.467256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.467262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.467401 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.467667 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.467671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.467673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.468100 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:24.468272 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:24.468468 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 05:11:24.468718 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":129,"threads_started":1}
I20260811 05:11:24.469497 16493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:24.469722 16493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:24.469780 16493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Initialized, Role: FOLLOWER
I20260811 05:11:24.470235 16493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:24.470372 16493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:24.470448 16493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:24.470538 16493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:24.471401 16493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:24.471745 16493 leader_election.cc:302] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f43a28146084c6a8722ce404097e29c; no voters:
I20260811 05:11:24.472009 16493 leader_election.cc:288] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:11:24.472359 16500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:24.472638 16500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 LEADER]: Becoming Leader. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Running, Role: LEADER
I20260811 05:11:24.473088 16500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:24.473157 16493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:24.474741 16501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:11:24.475183 16501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:24.475068 16499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f43a28146084c6a8722ce404097e29c. Latest consensus state: current_term: 1 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:11:24.475528 16499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:24.475603 16513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:24.478371 16513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:24.483788 16513 catalog_manager.cc:1347] Generated new cluster ID: 4bfe84061c4b42fd9db71970e3050139
I20260811 05:11:24.483865 16513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:24.494508 16513 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:11:24.495446 16513 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:24.505824 16513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Generated new TSK 0
I20260811 05:11:24.506381 16513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:11:24.537997 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:24.662379 16521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:24.662580 16521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:24.662614 16521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:24.666250 16521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:24.666296 16521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:24.666344 16521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:11:24.670997 16521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:24.671778 16521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:24.672595 16521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:24.681285 16530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:24.681326 16528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:24.681286 16527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:24.681548 16521 server_base.cc:1034] running on GCE node
I20260811 05:11:24.682037 16521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:24.682663 16521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:24.683841 16521 hybrid_clock.cc:648] HybridClock initialized: now 1786425084683836 us; error 59 us; skew 500 ppm
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:24.686375 16521 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:11:24.687216 16521 webserver.cc:460] Webserver started at http://127.30.88.193:34309/ using document root <none> and password file <none>
I20260811 05:11:24.687692 16521 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:24.687747 16521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:24.687918 16521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:24.689749 16521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "053dc20c44024980bb590b2aeb68baa9"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "cdfe8614d2bcb9971a7fee52ebe5d665"
server_key_iv: "473fbaeadd27bb790f8c418f26af09b6"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.690249 16521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "053dc20c44024980bb590b2aeb68baa9"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "cdfe8614d2bcb9971a7fee52ebe5d665"
server_key_iv: "473fbaeadd27bb790f8c418f26af09b6"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.693692 16521 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:11:24.695955 16538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.697093 16521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:24.697225 16521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "053dc20c44024980bb590b2aeb68baa9"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "cdfe8614d2bcb9971a7fee52ebe5d665"
server_key_iv: "473fbaeadd27bb790f8c418f26af09b6"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.697320 16521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:24.712558 16521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:24.715294 16521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:24.715461 16521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:24.715953 16521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:24.716678 16521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:24.716734 16521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.716795 16521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:24.716827 16521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.724817 16521 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:39929
I20260811 05:11:24.724888 16667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:39929 every 8 connection(s)
I20260811 05:11:24.725600 16521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:11:24.734548 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16521
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:24.734699 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:11:24.735054 31075 external_mini_cluster.cc:1428] Setting key e7d4ac3ef89693bd3055c478c1cffc4f
I20260811 05:11:24.739415 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:24.727344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:41587 (local address 127.30.88.254:35195)
0811 05:11:24.727483 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:24.727492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:24.728378 (+ 886us) server_negotiation.cc:407] Connection header received
0811 05:11:24.729643 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:24.729646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:24.729696 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:24.729779 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:24.730992 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.731542 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.732608 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.732843 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.732926 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:24.735423 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:24.735427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:24.735429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:24.735452 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:24.737071 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.737717 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.737720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.737721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.737792 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.738177 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.738179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.738180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.738403 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:24.738506 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:24.739141 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 05:11:24.739260 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:11:24.740079 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:24.727749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:41587)
0811 05:11:24.728213 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:24.728263 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:24.729425 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:24.729934 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:24.729940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:24.730308 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:24.730791 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:24.730804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.731652 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.731655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:24.732500 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:24.732510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.732972 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.732975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:24.733119 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:24.733671 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:24.733699 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:24.735257 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:24.737199 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:24.737207 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:24.737224 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:24.737615 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:24.737884 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:24.737889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:24.737894 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:24.738084 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:24.738527 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:24.738536 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:24.738830 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 05:11:24.739085 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 05:11:24.739367 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260811 05:11:24.741315 16668 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
I20260811 05:11:24.741554 16668 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:24.742071 16668 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:24.743793 16454 ts_manager.cc:194] Registered new tserver with Master: 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929)
I20260811 05:11:24.745440 16454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:41587
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:11:24.764093 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:24.868258 16673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:24.868497 16673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:24.868564 16673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:24.872519 16673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:24.872591 16673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:24.872668 16673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:11:24.878073 16673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:24.879035 16673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:24.879935 16673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:24.888805 16678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:24.888867 16673 server_base.cc:1034] running on GCE node
W20260811 05:11:24.888806 16681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:24.888806 16679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:24.889387 16673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:24.889915 16673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:24.891100 16673 hybrid_clock.cc:648] HybridClock initialized: now 1786425084891091 us; error 30 us; skew 500 ppm
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:24.893679 16673 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:11:24.894465 16673 webserver.cc:460] Webserver started at http://127.30.88.194:44913/ using document root <none> and password file <none>
I20260811 05:11:24.895095 16673 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:24.895181 16673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:24.895378 16673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:24.897217 16673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "a0ef0e46e435453db101c155ca7800ea"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "e12e49592868995c99cd5f41abeb58a1"
server_key_iv: "623266fcb597f01cf40282105fc5a244"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.897735 16673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ef0e46e435453db101c155ca7800ea"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "e12e49592868995c99cd5f41abeb58a1"
server_key_iv: "623266fcb597f01cf40282105fc5a244"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.901342 16673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:11:24.903564 16688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.904706 16673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:24.904831 16673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "a0ef0e46e435453db101c155ca7800ea"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "e12e49592868995c99cd5f41abeb58a1"
server_key_iv: "623266fcb597f01cf40282105fc5a244"
server_key_version: "encryptionkey@0"
I20260811 05:11:24.904935 16673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:24.924185 16673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:24.926716 16673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:24.926957 16673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:24.927466 16673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:24.928270 16673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:24.928332 16673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.928396 16673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:24.928442 16673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:24.936923 16673 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:35755
I20260811 05:11:24.936959 16814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:35755 every 8 connection(s)
I20260811 05:11:24.937661 16673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:11:24.939663 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16673
I20260811 05:11:24.939783 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:11:24.940099 31075 external_mini_cluster.cc:1428] Setting key cb0463730242b376b3e7756b81c1728b
Aug 11 05:11:24 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425084, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:11:24.951887 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:24.939244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:43009 (local address 127.30.88.254:35195)
0811 05:11:24.939365 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:24.939369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:24.940211 (+ 842us) server_negotiation.cc:407] Connection header received
0811 05:11:24.941624 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:24.941627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:24.941682 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:24.941771 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:24.942808 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.943594 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.945261 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.945609 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.945720 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:24.948112 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:24.948118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:24.948122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:24.948155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:24.949788 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.950287 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.950293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.950297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.950393 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:24.950760 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:24.950766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:24.950769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:24.951048 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:24.951149 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:24.951656 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 05:11:24.951755 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:11:24.952646 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:24.939625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:43009)
0811 05:11:24.940067 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:24.940104 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:24.941381 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:24.941932 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:24.941938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:24.942257 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:24.942658 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:24.942670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.943759 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.943765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:24.945114 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:24.945125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:24.945751 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:24.945754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:24.945885 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:24.946442 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:24.946459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:24.947946 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:24.949918 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:24.949922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:24.949932 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:24.950180 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:24.950501 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:24.950503 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:24.950504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:24.950654 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:24.951148 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:24.951153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:24.951408 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 05:11:24.951564 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 05:11:24.951791 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":132,"threads_started":1}
I20260811 05:11:24.953769 16815 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:11:24.954020 16815 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:24.954577 16815 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:24.955771 16454 ts_manager.cc:194] Registered new tserver with Master: a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:24.956595 16454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:43009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:11:24.967402 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:25.077023 16820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:25.077234 16820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:25.077273 16820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:25.081079 16820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:25.081137 16820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:25.081194 16820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:11:25.086215 16820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:25.087162 16820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:25.088050 16820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:25.097213 16825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:25.097239 16828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:25.097311 16820 server_base.cc:1034] running on GCE node
W20260811 05:11:25.097239 16826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:25.097831 16820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:25.098423 16820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:25.099609 16820 hybrid_clock.cc:648] HybridClock initialized: now 1786425085099594 us; error 34 us; skew 500 ppm
Aug 11 05:11:25 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425085, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:25.102509 16820 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:11:25.103499 16820 webserver.cc:460] Webserver started at http://127.30.88.195:35205/ using document root <none> and password file <none>
I20260811 05:11:25.104038 16820 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:25.104125 16820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:25.104451 16820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:11:25.106266 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "a0f904223bb94df9838dc6a211b39e9f"
format_stamp: "Formatted at 2026-08-11 05:11:25 on dist-test-slave-7c35"
server_key: "d80c79d199afb8bbadc3b6adac7b0a37"
server_key_iv: "46295a0333b7347b1e2290c1c3bc0b0e"
server_key_version: "encryptionkey@0"
I20260811 05:11:25.106766 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "a0f904223bb94df9838dc6a211b39e9f"
format_stamp: "Formatted at 2026-08-11 05:11:25 on dist-test-slave-7c35"
server_key: "d80c79d199afb8bbadc3b6adac7b0a37"
server_key_iv: "46295a0333b7347b1e2290c1c3bc0b0e"
server_key_version: "encryptionkey@0"
I20260811 05:11:25.110200 16820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:11:25.112526 16836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:25.113678 16820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:25.113795 16820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "a0f904223bb94df9838dc6a211b39e9f"
format_stamp: "Formatted at 2026-08-11 05:11:25 on dist-test-slave-7c35"
server_key: "d80c79d199afb8bbadc3b6adac7b0a37"
server_key_iv: "46295a0333b7347b1e2290c1c3bc0b0e"
server_key_version: "encryptionkey@0"
I20260811 05:11:25.113895 16820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:25.142045 16820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:25.144800 16820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:25.145002 16820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:25.145596 16820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:25.146380 16820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:25.146428 16820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:25.146497 16820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:25.146536 16820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:25.156448 16820 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39735
I20260811 05:11:25.156471 16959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39735 every 8 connection(s)
I20260811 05:11:25.157244 16820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:11:25.164083 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 16820
I20260811 05:11:25.164208 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:11:25.164542 31075 external_mini_cluster.cc:1428] Setting key f22653fbb385929187e99c878651201d
Aug 11 05:11:25 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425085, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:25.170092 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.158956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:45687 (local address 127.30.88.254:35195)
0811 05:11:25.159093 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.159096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.159801 (+ 705us) server_negotiation.cc:407] Connection header received
0811 05:11:25.160981 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.160984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.161044 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.161145 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:25.162310 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.162801 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.163920 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.164126 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.164233 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.167225 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:25.167228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:25.167230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:25.167251 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:25.168458 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:25.168900 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:25.168902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:25.168903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:25.168969 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:25.169250 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:25.169252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:25.169253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:25.169400 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:25.169468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.169879 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.169971 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:11:25.170809 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.159252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:45687)
0811 05:11:25.159670 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.159708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.160779 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.161362 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.161369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.161733 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:25.162175 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.162187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.162976 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.162979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.163789 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.163798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.164361 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.164368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.164578 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.165274 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:25.165291 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:25.167078 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:25.168560 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:25.168564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:25.168573 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:25.168805 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:25.169046 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:25.169048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:25.169049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:25.169172 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:25.169491 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:25.169495 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:25.169688 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.169836 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.170046 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":126,"threads_started":1}
I20260811 05:11:25.171911 16960 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
I20260811 05:11:25.172107 16960 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:25.172461 16960 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:25.173302 16454 ts_manager.cc:194] Registered new tserver with Master: a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735)
I20260811 05:11:25.173872 16454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:45687
I20260811 05:11:25.178534 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:11:25.187351 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.180574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.30.88.254:35195)
0811 05:11:25.180694 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.180697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.180938 (+ 241us) server_negotiation.cc:407] Connection header received
0811 05:11:25.181041 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.181043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.181083 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.181147 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:25.181835 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.182304 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.183187 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.183381 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.183454 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.184652 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:25.184655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:25.184657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:25.184676 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:25.185959 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:25.186324 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:25.186326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:25.186327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:25.186389 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:25.186614 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:25.186616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:25.186617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:25.186793 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:25.186934 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.187143 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.187245 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:11:25.189778 16454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:11:25.192735 16454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:11:25.206642 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.201100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.30.88.193:39929)
0811 05:11:25.202482 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.202497 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.202517 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:11:25.202589 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.202593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.202769 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.202929 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:25.203139 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203812 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.205204 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.206008 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.206137 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.206284 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.206395 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.206491 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1264,"thread_start_us":80,"threads_started":1}
I20260811 05:11:25.206637 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.200542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39735 (local address 127.0.0.1:47642)
0811 05:11:25.201410 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.201423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.201511 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.202265 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.202271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.202294 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:25.202429 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.202436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203872 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.203875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.205305 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.205316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.206299 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.206302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.206386 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.206404 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.206464 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.206513 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":103,"thread_start_us":108,"threads_started":1}
I20260811 05:11:25.206636 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.200460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:39929 (local address 127.0.0.1:54526)
0811 05:11:25.200770 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.200813 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.200929 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.202909 (+ 1980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.202911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.202929 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:25.203030 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.203036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203933 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.203936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.205041 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.205051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.206138 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.206140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.206219 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.206272 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.206370 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.206497 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":35,"thread_start_us":79,"threads_started":1}
I20260811 05:11:25.206957 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.201252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.30.88.195:39735)
0811 05:11:25.201819 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.201832 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.201855 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:11:25.201952 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.201958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.202117 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.202235 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:25.202574 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203699 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.205435 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.206192 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.206286 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.206405 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.206704 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.206792 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":155,"threads_started":1}
I20260811 05:11:25.207899 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.200888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:35755 (local address 127.0.0.1:44706)
0811 05:11:25.201493 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.201698 (+ 205us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.201799 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.202472 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.202476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.202497 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:25.202596 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.202602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203846 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.203849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.205304 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.205316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.207531 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.207534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.207651 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.207673 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.207721 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.207785 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":155,"thread_start_us":101,"threads_started":1}
I20260811 05:11:25.208783 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.201668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.30.88.194:35755)
0811 05:11:25.202035 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.202044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.202070 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:11:25.202130 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.202133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.202312 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.202467 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:25.202701 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.203704 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.206459 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.207404 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.208426 (+ 1022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.208545 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.208588 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.208662 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":103,"threads_started":1}
I20260811 05:11:25.209612 16889 tablet_service.cc:1467] Processing CreateTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 (DEFAULT_TABLE table=test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:25.209611 16591 tablet_service.cc:1467] Processing CreateTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 (DEFAULT_TABLE table=test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:25.210544 16591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 728cf345d5aa471f9ed28dbdd8eba133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:25.210554 16889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 728cf345d5aa471f9ed28dbdd8eba133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:25.213488 16741 tablet_service.cc:1467] Processing CreateTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 (DEFAULT_TABLE table=test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:25.214226 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 728cf345d5aa471f9ed28dbdd8eba133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:25.216195 16981 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Bootstrap starting.
I20260811 05:11:25.218612 16981 tablet_bootstrap.cc:654] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:25.219470 16982 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Bootstrap starting.
I20260811 05:11:25.219455 16981 log.cc:824] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:25.220791 16984 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Bootstrap starting.
I20260811 05:11:25.221418 16981 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: No bootstrap required, opened a new log
I20260811 05:11:25.221477 16982 tablet_bootstrap.cc:654] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:25.221639 16981 ts_tablet_manager.cc:1397] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 05:11:25.222663 16982 log.cc:824] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:25.223253 16984 tablet_bootstrap.cc:654] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:25.224000 16984 log.cc:824] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:25.224524 16982 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: No bootstrap required, opened a new log
I20260811 05:11:25.224745 16982 ts_tablet_manager.cc:1397] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:11:25.224861 16981 raft_consensus.cc:348] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.225102 16981 raft_consensus.cc:374] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:25.225169 16981 raft_consensus.cc:729] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f904223bb94df9838dc6a211b39e9f, State: Initialized, Role: FOLLOWER
I20260811 05:11:25.225572 16981 consensus_queue.cc:260] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.225826 16984 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: No bootstrap required, opened a new log
I20260811 05:11:25.226059 16984 ts_tablet_manager.cc:1397] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260811 05:11:25.226191 16960 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
I20260811 05:11:25.226523 16981 ts_tablet_manager.cc:1428] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 05:11:25.228899 16982 raft_consensus.cc:348] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.229174 16982 raft_consensus.cc:374] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:25.229249 16982 raft_consensus.cc:729] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053dc20c44024980bb590b2aeb68baa9, State: Initialized, Role: FOLLOWER
I20260811 05:11:25.229665 16982 consensus_queue.cc:260] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.229816 16984 raft_consensus.cc:348] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.230070 16984 raft_consensus.cc:374] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:25.230142 16984 raft_consensus.cc:729] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ef0e46e435453db101c155ca7800ea, State: Initialized, Role: FOLLOWER
I20260811 05:11:25.230522 16984 consensus_queue.cc:260] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.230628 16982 ts_tablet_manager.cc:1428] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:11:25.231087 16668 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
I20260811 05:11:25.231262 16815 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
I20260811 05:11:25.231441 16984 ts_tablet_manager.cc:1428] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:11:25.333187 16987 raft_consensus.cc:482] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:25.333367 16987 raft_consensus.cc:504] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.334545 16987 leader_election.cc:288] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:25.338394 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.334767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:35755 (local address 127.30.88.195:32809)
0811 05:11:25.334943 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.334959 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.335053 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.335299 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.335303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.335325 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:25.335430 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.335435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.336321 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.336324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.337206 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.337215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.337956 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.337960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.338067 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.338094 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.338181 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.338254 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 05:11:25.338425 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.334935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:32809 (local address 127.30.88.194:35755)
0811 05:11:25.335070 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.335073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.335088 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:11:25.335136 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.335138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.335178 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.335276 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:25.335566 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.336191 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.337351 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.337838 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.337929 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.337976 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.338226 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.338297 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:11:25.338666 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.334768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:39929 (local address 127.30.88.195:40757)
0811 05:11:25.335407 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.335418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:25.335530 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:25.335744 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:25.335747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:25.335766 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:25.335874 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.335880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.336682 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.336685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.337524 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:25.337534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.338245 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.338248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:25.338339 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.338361 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:11:25.338449 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:11:25.338519 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":221,"thread_start_us":173,"threads_started":1}
I20260811 05:11:25.338711 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.334860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:40757 (local address 127.30.88.193:39929)
0811 05:11:25.335234 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.335238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.335516 (+ 278us) server_negotiation.cc:407] Connection header received
0811 05:11:25.335612 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.335614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.335655 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.335709 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:25.335980 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.336564 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.337650 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.338124 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.338229 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.338282 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.338478 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.338563 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260811 05:11:25.339310 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "728cf345d5aa471f9ed28dbdd8eba133" candidate_uuid: "a0f904223bb94df9838dc6a211b39e9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ef0e46e435453db101c155ca7800ea" is_pre_election: true
I20260811 05:11:25.339500 16614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "728cf345d5aa471f9ed28dbdd8eba133" candidate_uuid: "a0f904223bb94df9838dc6a211b39e9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053dc20c44024980bb590b2aeb68baa9" is_pre_election: true
I20260811 05:11:25.339624 16765 raft_consensus.cc:2454] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f904223bb94df9838dc6a211b39e9f in term 0.
I20260811 05:11:25.339818 16614 raft_consensus.cc:2454] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f904223bb94df9838dc6a211b39e9f in term 0.
I20260811 05:11:25.340167 16841 leader_election.cc:302] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f; no voters:
I20260811 05:11:25.340466 16987 raft_consensus.cc:2781] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:11:25.340540 16987 raft_consensus.cc:482] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:11:25.340569 16987 raft_consensus.cc:3037] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:25.341601 16987 raft_consensus.cc:504] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.341960 16987 leader_election.cc:288] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [CANDIDATE]: Term 1 election: Requested vote from peers 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:25.342329 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "728cf345d5aa471f9ed28dbdd8eba133" candidate_uuid: "a0f904223bb94df9838dc6a211b39e9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ef0e46e435453db101c155ca7800ea"
I20260811 05:11:25.342346 16614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "728cf345d5aa471f9ed28dbdd8eba133" candidate_uuid: "a0f904223bb94df9838dc6a211b39e9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053dc20c44024980bb590b2aeb68baa9"
I20260811 05:11:25.342428 16765 raft_consensus.cc:3037] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:25.342475 16614 raft_consensus.cc:3037] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:25.343468 16614 raft_consensus.cc:2454] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f904223bb94df9838dc6a211b39e9f in term 1.
I20260811 05:11:25.343537 16765 raft_consensus.cc:2454] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0f904223bb94df9838dc6a211b39e9f in term 1.
I20260811 05:11:25.343806 16837 leader_election.cc:302] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9, a0f904223bb94df9838dc6a211b39e9f; no voters:
I20260811 05:11:25.344058 16987 raft_consensus.cc:2781] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:25.344287 16987 raft_consensus.cc:686] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 1 LEADER]: Becoming Leader. State: Replica: a0f904223bb94df9838dc6a211b39e9f, State: Running, Role: LEADER
I20260811 05:11:25.344631 16987 consensus_queue.cc:237] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:25.347759 16453 catalog_manager.cc:5495] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f reported cstate change: term changed from 0 to 1, leader changed from <none> to a0f904223bb94df9838dc6a211b39e9f (127.30.88.195). New cstate: current_term: 1 leader_uuid: "a0f904223bb94df9838dc6a211b39e9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } health_report { overall_health: UNKNOWN } } }
I20260811 05:11:25.392305 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.388962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.30.88.195:39735)
0811 05:11:25.389114 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:25.389118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:25.389130 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:11:25.389204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:25.389206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:25.389257 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:25.389352 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:25.389555 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.390052 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.390924 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:25.391220 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:25.391299 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:25.391581 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:25.391987 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:11:25.392084 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:25.392136 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 05:11:25.392183 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:11:25.398187 16614 raft_consensus.cc:1261] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Refusing update from remote peer a0f904223bb94df9838dc6a211b39e9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:25.398186 16765 raft_consensus.cc:1261] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Refusing update from remote peer a0f904223bb94df9838dc6a211b39e9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:25.398893 16992 consensus_queue.cc:1035] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:25.399075 16987 consensus_queue.cc:1035] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:25.408805 16998 mvcc.cc:205] Tried to move back new op lower bound from 7317197149783285760 to 7317197149574262784. Current Snapshot: MvccSnapshot[applied={T|T < 7317197149783285760 or (T in {7317197149783285760})}]
I20260811 05:11:25.415493 16454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:25.415663 16454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:25.417999 16454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Sending DeleteTablet for 3 replicas of tablet 728cf345d5aa471f9ed28dbdd8eba133
I20260811 05:11:25.418728 16889 tablet_service.cc:1514] Processing DeleteTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:25 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:47642
I20260811 05:11:25.418749 16591 tablet_service.cc:1514] Processing DeleteTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:25 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:54526
I20260811 05:11:25.418783 16741 tablet_service.cc:1514] Processing DeleteTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:25 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:44706
I20260811 05:11:25.419318 17010 tablet_replica.cc:330] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: stopping tablet replica
I20260811 05:11:25.419330 17007 tablet_replica.cc:330] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: stopping tablet replica
I20260811 05:11:25.419353 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16521
I20260811 05:11:25.419580 17007 raft_consensus.cc:2229] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:25.426143 17009 tablet_replica.cc:330] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: stopping tablet replica
I20260811 05:11:25.426357 17009 raft_consensus.cc:2229] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:25.426573 17009 raft_consensus.cc:2258] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:25.431622 17010 raft_consensus.cc:2229] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:11:25.432039 17010 raft_consensus.cc:2258] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:25.433399 17009 ts_tablet_manager.cc:1905] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:25.433408 17010 ts_tablet_manager.cc:1905] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:25.436049 17009 ts_tablet_manager.cc:1918] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:25.436216 17009 log.cc:1196] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/728cf345d5aa471f9ed28dbdd8eba133
I20260811 05:11:25.436600 17009 ts_tablet_manager.cc:1939] T 728cf345d5aa471f9ed28dbdd8eba133 P a0ef0e46e435453db101c155ca7800ea: Deleting consensus metadata
I20260811 05:11:25.436846 17010 ts_tablet_manager.cc:1918] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:25.437326 17010 log.cc:1196] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/728cf345d5aa471f9ed28dbdd8eba133
I20260811 05:11:25.437745 17010 ts_tablet_manager.cc:1939] T 728cf345d5aa471f9ed28dbdd8eba133 P a0f904223bb94df9838dc6a211b39e9f: Deleting consensus metadata
I20260811 05:11:25.437863 16439 catalog_manager.cc:4841] TS a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): tablet 728cf345d5aa471f9ed28dbdd8eba133 (table test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]) successfully deleted
I20260811 05:11:25.438968 16436 catalog_manager.cc:4841] TS a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): tablet 728cf345d5aa471f9ed28dbdd8eba133 (table test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]) successfully deleted
W20260811 05:11:25.439357 16435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:39929 (error 108)
I20260811 05:11:25.439631 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16673
I20260811 05:11:25.440946 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:25.440630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:39929 (local address 127.0.0.1:54542)
0811 05:11:25.440765 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:25.440858 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:39929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 05:11:25.441128 16435 catalog_manager.cc:4584] TS 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 728cf345d5aa471f9ed28dbdd8eba133: Network error: Client connection negotiation failed: client connection to 127.30.88.193:39929: connect: Connection refused (error 111)
I20260811 05:11:25.447144 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16820
I20260811 05:11:25.454131 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 16418
I20260811 05:11:45.461288 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--webserver_interface=127.30.88.254
--webserver_port=35015
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:11:45.571934 17012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:45.572218 17012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:45.572304 17012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:45.576200 17012 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:11:45.576287 17012 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:11:45.576329 17012 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:11:45.576398 17012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:11:45.576452 17012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:45.576491 17012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:45.576548 17012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:11:45.576591 17012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:11:45.581974 17012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=35015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:45.583038 17012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:45.584018 17012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:11:45.597354 17012 server_base.cc:1034] running on GCE node
W20260811 05:11:45.597327 17018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.597342 17019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.597543 17022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:45.598183 17012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:45.599335 17012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:45.600553 17012 hybrid_clock.cc:648] HybridClock initialized: now 1786425105600540 us; error 29 us; skew 500 ppm
Aug 11 05:11:45 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:45.603433 17012 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:11:45.604230 17012 webserver.cc:460] Webserver started at http://127.30.88.254:35015/ using document root <none> and password file <none>
I20260811 05:11:45.604753 17012 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:45.604823 17012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:45.608507 17012 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:11:45.610502 17031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:45.611788 17012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:45.611954 17012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "2f43a28146084c6a8722ce404097e29c"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "84b050e19b041a95d80b31a936127954"
server_key_iv: "2d9e95e24f503ae9d4eb8c5a06b8fa6b"
server_key_version: "encryptionkey@0"
I20260811 05:11:45.612491 17012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:45.627115 17012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:45.630579 17012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:45.630728 17012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:45.637401 17012 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35195
I20260811 05:11:45.637660 17089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35195 every 8 connection(s)
I20260811 05:11:45.638500 17012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:11:45.642661 17091 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:11:45.644348 17091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap starting.
I20260811 05:11:45.648399 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17012
I20260811 05:11:45.648857 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:39929
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=34309
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:11:45.661438 17091 log.cc:824] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:45.669821 17091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:45.670193 17091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap complete.
I20260811 05:11:45.674117 17091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:45.674517 17091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Initialized, Role: FOLLOWER
I20260811 05:11:45.675057 17091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:45.675194 17091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:11:45.675302 17091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:11:45.675417 17091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:11:45.677208 17091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:45.677551 17091 leader_election.cc:302] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f43a28146084c6a8722ce404097e29c; no voters:
I20260811 05:11:45.677843 17091 leader_election.cc:288] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:11:45.677964 17096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:11:45.678194 17096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 LEADER]: Becoming Leader. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Running, Role: LEADER
I20260811 05:11:45.678553 17096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:11:45.679105 17091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:11:45.680334 17098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f43a28146084c6a8722ce404097e29c. Latest consensus state: current_term: 2 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:11:45.680365 17097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:11:45.680516 17098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:45.680533 17097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:11:45.680892 17105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:11:45.685602 17105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:11:45.686408 17105 catalog_manager.cc:1259] Loaded cluster ID: 4bfe84061c4b42fd9db71970e3050139
I20260811 05:11:45.686470 17105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:11:45.688918 17105 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:11:45.689589 17105 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Loaded TSK: 0
I20260811 05:11:45.711324 17105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Generated new TSK 1
I20260811 05:11:45.711567 17105 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:11:45.765993 17093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:45.766222 17093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:45.766263 17093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:45.769979 17093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:45.770027 17093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:45.770108 17093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:11:45.774914 17093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:39929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=34309
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:45.775764 17093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:45.776701 17093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:45.786096 17120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.786082 17121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.786207 17123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:45.786669 17093 server_base.cc:1034] running on GCE node
I20260811 05:11:45.787097 17093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:45.787755 17093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:45.788920 17093 hybrid_clock.cc:648] HybridClock initialized: now 1786425105788906 us; error 35 us; skew 500 ppm
Aug 11 05:11:45 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:45.791579 17093 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:11:45.792461 17093 webserver.cc:460] Webserver started at http://127.30.88.193:34309/ using document root <none> and password file <none>
I20260811 05:11:45.793102 17093 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:45.793203 17093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:45.796989 17093 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:11:45.798954 17131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:45.800120 17093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:45.800226 17093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "053dc20c44024980bb590b2aeb68baa9"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "cdfe8614d2bcb9971a7fee52ebe5d665"
server_key_iv: "473fbaeadd27bb790f8c418f26af09b6"
server_key_version: "encryptionkey@0"
I20260811 05:11:45.800737 17093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:45.816375 17093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:45.819747 17093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:45.819943 17093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:45.820510 17093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:45.821556 17139 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:11:45.825536 17093 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:11:45.825589 17093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260811 05:11:45.825652 17093 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:11:45.827268 17093 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:11:45.827311 17093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:45.827610 17139 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Bootstrap starting.
I20260811 05:11:45.837852 17093 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:39929
I20260811 05:11:45.837944 17258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:39929 every 8 connection(s)
I20260811 05:11:45.838788 17093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:11:45.845819 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17093
I20260811 05:11:45.846067 17139 log.cc:824] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:45.846316 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:35755
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=44913
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:11:45 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:45.853868 17139 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:11:45.854328 17139 tablet_bootstrap.cc:492] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Bootstrap complete.
I20260811 05:11:45.855629 17139 ts_tablet_manager.cc:1397] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Time spent bootstrapping tablet: real 0.028s user 0.023s sys 0.004s
I20260811 05:11:45.858124 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:45.841064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:36329 (local address 127.30.88.254:35195)
0811 05:11:45.841530 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:45.841548 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:45.841633 (+ 85us) server_negotiation.cc:407] Connection header received
0811 05:11:45.843174 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:45.843185 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:45.843449 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:45.843701 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:45.845198 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.846602 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.848256 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.849001 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.849182 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:45.852076 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:45.852082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:45.852096 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:45.852127 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:45.854345 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:45.855079 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:45.855088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:45.855096 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:45.855245 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:45.855734 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:45.855737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:45.855739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:45.856180 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:45.856320 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:45.857032 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 05:11:45.857302 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":121,"threads_started":1}
I20260811 05:11:45.858188 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:45.840898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:36329)
0811 05:11:45.841479 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:45.841534 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:45.842745 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:45.843899 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:45.843911 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:45.844396 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:45.845032 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:45.845051 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.846763 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.846767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:45.848111 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:45.848120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.849162 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.849166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:45.849340 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:45.849912 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:45.849930 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:45.851849 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:45.854510 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:45.854519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:45.854537 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:45.854942 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:45.855353 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:45.855359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:45.855363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:45.855571 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:45.856317 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:45.856327 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:45.856654 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 05:11:45.856876 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 05:11:45.857214 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":143,"threads_started":1}
I20260811 05:11:45.858286 17139 raft_consensus.cc:348] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:45.858989 17139 raft_consensus.cc:729] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053dc20c44024980bb590b2aeb68baa9, State: Initialized, Role: FOLLOWER
I20260811 05:11:45.859619 17139 consensus_queue.cc:260] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } } peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } }
I20260811 05:11:45.860630 17139 ts_tablet_manager.cc:1428] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:11:45.861150 17259 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
I20260811 05:11:45.861403 17259 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:45.861855 17259 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:45.864040 17052 ts_manager.cc:194] Registered new tserver with Master: 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929)
I20260811 05:11:45.867399 17052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:36329
I20260811 05:11:45.869710 17259 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
Aug 11 05:11:45 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.193@KRBTEST.COM
I20260811 05:11:45.876801 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:45.866599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.30.88.193:39929)
0811 05:11:45.867280 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:45.867290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:45.867311 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:11:45.867371 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:45.867376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:45.867561 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:45.867686 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:45.868383 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.869592 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.870579 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.870988 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.871100 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:45.873474 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:45.873482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:45.873491 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:45.873511 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:45.875083 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:45.875596 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:45.875600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:45.875601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:45.875674 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:45.875940 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:45.875942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:45.875943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:45.876137 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:45.876232 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:45.876573 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 05:11:45.876691 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":133,"threads_started":1}
I20260811 05:11:45.876824 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:45.866537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:39929 (local address 127.0.0.1:60012)
0811 05:11:45.866980 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:45.867022 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:45.867134 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:45.867715 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:45.867722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:45.868133 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:45.868257 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:45.868266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.869726 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.869733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:45.870452 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:45.870463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:45.871113 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:45.871116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:45.871252 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:45.871764 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:45.871775 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:45.873345 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:45.875188 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:45.875197 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:45.875202 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:45.875490 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:45.875758 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:45.875761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:45.875763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:45.875855 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:45.876241 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:45.876246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:45.876358 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 05:11:45.876504 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 05:11:45.876677 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260811 05:11:45.877748 17185 tablet_service.cc:1514] Processing DeleteTablet for tablet 728cf345d5aa471f9ed28dbdd8eba133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:25 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60012
I20260811 05:11:45.878662 17268 tablet_replica.cc:330] stopping tablet replica
I20260811 05:11:45.879024 17268 raft_consensus.cc:2229] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:45.879331 17268 raft_consensus.cc:2258] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:45.880359 17268 ts_tablet_manager.cc:1905] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:45.884110 17268 ts_tablet_manager.cc:1918] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:45.884234 17268 log.cc:1196] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal/wals/728cf345d5aa471f9ed28dbdd8eba133
I20260811 05:11:45.884506 17268 ts_tablet_manager.cc:1939] T 728cf345d5aa471f9ed28dbdd8eba133 P 053dc20c44024980bb590b2aeb68baa9: Deleting consensus metadata
I20260811 05:11:45.886101 17032 catalog_manager.cc:4841] TS 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929): tablet 728cf345d5aa471f9ed28dbdd8eba133 (table test-table [id=d5a14c24bf8b4c92807b30b63cb53b26]) successfully deleted
W20260811 05:11:45.963380 17264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:45.963609 17264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:45.963649 17264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:45.967368 17264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:45.967411 17264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:45.967460 17264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:11:45.972218 17264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:35755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=44913
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:45.973093 17264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:45.973975 17264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:45.983893 17274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.983908 17275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:45.983901 17277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:45.984228 17264 server_base.cc:1034] running on GCE node
I20260811 05:11:45.984782 17264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:45.985384 17264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:45.986553 17264 hybrid_clock.cc:648] HybridClock initialized: now 1786425105986541 us; error 23 us; skew 500 ppm
Aug 11 05:11:45 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:45.989270 17264 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:11:45.990092 17264 webserver.cc:460] Webserver started at http://127.30.88.194:44913/ using document root <none> and password file <none>
I20260811 05:11:45.990600 17264 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:45.990674 17264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:45.994370 17264 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:45.996381 17284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:45.997540 17264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:11:45.997653 17264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "a0ef0e46e435453db101c155ca7800ea"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "e12e49592868995c99cd5f41abeb58a1"
server_key_iv: "623266fcb597f01cf40282105fc5a244"
server_key_version: "encryptionkey@0"
I20260811 05:11:45.998139 17264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:46.017175 17264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:46.020300 17264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:46.020442 17264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:46.021018 17264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:46.021834 17264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:46.021881 17264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:46.021945 17264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:46.021983 17264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:46.030699 17264 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:35755
I20260811 05:11:46.030726 17410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:35755 every 8 connection(s)
I20260811 05:11:46.031687 17264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:11:46.035347 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17264
I20260811 05:11:46.035784 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:39735
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=35205
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:11:46 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425105, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:46.045864 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.033433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:34927 (local address 127.30.88.254:35195)
0811 05:11:46.033571 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.033575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.034378 (+ 803us) server_negotiation.cc:407] Connection header received
0811 05:11:46.035688 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.035692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.035757 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.035859 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:46.036998 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.037543 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.038675 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.039154 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.039277 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.041788 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:46.041794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:46.041799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:46.041841 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:46.043693 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.044195 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.044201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.044204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.044305 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.044620 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.044625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.044628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.044886 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:46.045005 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.045465 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.045624 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:11:46.046430 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.033792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:34927)
0811 05:11:46.034232 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.034271 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.035472 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.036038 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.036044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.036371 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:46.036847 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.036860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.037678 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.037682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.038545 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.038555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.039324 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.039327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.039470 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.040074 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:46.040090 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:46.041597 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:46.043809 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:46.043814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:46.043824 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:46.044080 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:46.044403 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:46.044405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:46.044406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:46.044526 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:46.044996 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:46.045001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:46.045206 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.045375 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.045603 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260811 05:11:46.047610 17411 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
I20260811 05:11:46.047870 17411 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:46.048478 17411 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:46.049548 17052 ts_manager.cc:194] Registered new tserver with Master: a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:46.050303 17052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:34927
W20260811 05:11:46.148201 17415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:11:46.148419 17415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:11:46.148459 17415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:11:46.152338 17415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:11:46.152385 17415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:11:46.152439 17415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:11:46.157425 17415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:39735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=35205
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:11:46.158327 17415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:11:46.159327 17415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:11:46.168565 17422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:11:46.168569 17421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:46.168711 17415 server_base.cc:1034] running on GCE node
W20260811 05:11:46.168562 17425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:11:46.169245 17415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:11:46.169831 17415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:11:46.171041 17415 hybrid_clock.cc:648] HybridClock initialized: now 1786425106171032 us; error 31 us; skew 500 ppm
Aug 11 05:11:46 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425106, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:11:46.173754 17415 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:11:46.174572 17415 webserver.cc:460] Webserver started at http://127.30.88.195:35205/ using document root <none> and password file <none>
I20260811 05:11:46.175163 17415 fs_manager.cc:359] Metadata directory not provided
I20260811 05:11:46.175254 17415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:11:46.178642 17415 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:11:46.180927 17432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:46.182163 17415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:11:46.182292 17415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "a0f904223bb94df9838dc6a211b39e9f"
format_stamp: "Formatted at 2026-08-11 05:11:25 on dist-test-slave-7c35"
server_key: "d80c79d199afb8bbadc3b6adac7b0a37"
server_key_iv: "46295a0333b7347b1e2290c1c3bc0b0e"
server_key_version: "encryptionkey@0"
I20260811 05:11:46.182806 17415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:11:46.192768 17415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:11:46.195398 17415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:11:46.195592 17415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:11:46.196146 17415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:11:46.196913 17415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:11:46.196985 17415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:46.197048 17415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:11:46.197084 17415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:11:46.205966 17415 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39735
I20260811 05:11:46.205998 17558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39735 every 8 connection(s)
I20260811 05:11:46.206905 17415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:11:46.213066 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17415
Aug 11 05:11:46 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425106, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:11:46.221156 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.208585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:47881 (local address 127.30.88.254:35195)
0811 05:11:46.208721 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.208724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.209883 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 05:11:46.210909 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.210912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.210978 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.211085 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:46.212172 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.212731 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.214027 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.214398 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.214490 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.217120 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:46.217123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:46.217125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:46.217148 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:46.218761 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.219412 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.219417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.219419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.219513 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.219931 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.219935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.219937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.220120 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:46.220199 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.220831 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.220953 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:11:46.221971 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.208872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:47881)
0811 05:11:46.209741 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.209779 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.210634 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.211257 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.211264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.211589 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:11:46.212005 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.212018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.212860 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.212864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.213899 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.213913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.214550 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.214554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.214786 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.215412 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:11:46.215427 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:11:46.216956 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:11:46.218880 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:46.218886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:46.218899 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:46.219282 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:46.219618 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:11:46.219622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:11:46.219625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:11:46.219808 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:11:46.220234 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:11:46.220240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:11:46.220551 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.220748 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.221005 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":567,"threads_started":1}
I20260811 05:11:46.222398 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.213717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.30.88.254:35195)
0811 05:11:46.213937 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.213940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.214101 (+ 161us) server_negotiation.cc:407] Connection header received
0811 05:11:46.214206 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.214209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.214250 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.214339 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:46.215317 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.215843 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.216825 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.217183 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.217257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.218886 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:46.218890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:46.218893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:46.218918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:46.220578 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.221094 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.221097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.221099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.221186 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.221499 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.221506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.221508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.221683 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:46.221853 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.222116 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.222242 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":58,"threads_started":1}
I20260811 05:11:46.223145 17559 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35195
I20260811 05:11:46.223369 17559 heartbeater.cc:464] Registering TS with master...
I20260811 05:11:46.223819 17559 heartbeater.cc:511] Master 127.30.88.254:35195 requested a full tablet report, sending...
I20260811 05:11:46.224819 17052 ts_manager.cc:194] Registered new tserver with Master: a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735)
I20260811 05:11:46.225348 17052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:47881
I20260811 05:11:46.236459 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:11:46.239969 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.237048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.30.88.254:35195)
0811 05:11:46.237165 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.237169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.237325 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:11:46.237429 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.237431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.237507 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.237591 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:46.237802 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.238261 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.239055 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.239257 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.239324 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.239583 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:46.239683 (+ 100us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 05:11:46.239865 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38222: token expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 05:11:46.240029 17564 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38222: token expired
W20260811 05:11:46.240370 16966 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.88.254:35195)
I20260811 05:11:46.246598 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.240666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.30.88.254:35195)
0811 05:11:46.240770 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.240774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.240788 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:11:46.240865 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.240866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.240895 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.240951 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:11:46.241411 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.241813 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.242506 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.242679 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.242742 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.244063 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:11:46.244066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:11:46.244067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:11:46.244087 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:11:46.245278 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.245638 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.245640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.245641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.245709 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:11:46.245972 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:11:46.245980 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:11:46.245981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:11:46.246136 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:11:46.246226 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.246423 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.246506 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:11:46.248891 17052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:11:46.252018 17052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:11:46.259754 17185 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab91c6406c24f48890affdde83502ab (DEFAULT_TABLE table=test-table [id=81fe044a39f24219a3851c5e0d136d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:46.260401 17185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab91c6406c24f48890affdde83502ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:46.263334 17139 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Bootstrap starting.
I20260811 05:11:46.263571 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.258645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39735 (local address 127.0.0.1:40584)
0811 05:11:46.258758 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.258785 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.258895 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.259762 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.259765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.259788 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:46.259873 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.259878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.261278 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.261282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.262122 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.262132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.263119 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.263123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.263234 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.263288 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.263373 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.263454 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":39}
I20260811 05:11:46.263695 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.258859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.30.88.195:39735)
0811 05:11:46.259252 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.259272 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.259296 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:11:46.259377 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.259385 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.259608 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.259762 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:46.260007 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.261165 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.262249 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.262949 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.263104 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.263260 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.263427 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.263525 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":207,"threads_started":1}
I20260811 05:11:46.264144 17139 tablet_bootstrap.cc:654] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:46.264796 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.258948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:35755 (local address 127.0.0.1:33984)
0811 05:11:46.259333 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.259345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.259423 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.260092 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.260095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.260112 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:46.260215 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.260223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.261410 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.261413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.262467 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.262478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.264361 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.264365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.264477 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.264511 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.264590 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.264660 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260811 05:11:46.264835 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.259357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.30.88.194:35755)
0811 05:11:46.259681 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.259695 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.259715 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:11:46.259783 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.259787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.259960 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.260099 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:46.260360 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.261301 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.262609 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.264215 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.264328 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.264450 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.264625 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.264709 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":136,"threads_started":1}
I20260811 05:11:46.266206 17139 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: No bootstrap required, opened a new log
I20260811 05:11:46.266306 17139 ts_tablet_manager.cc:1397] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:11:46.266770 17139 raft_consensus.cc:348] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.266913 17139 raft_consensus.cc:374] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:46.266959 17139 raft_consensus.cc:729] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053dc20c44024980bb590b2aeb68baa9, State: Initialized, Role: FOLLOWER
I20260811 05:11:46.266536 17485 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab91c6406c24f48890affdde83502ab (DEFAULT_TABLE table=test-table [id=81fe044a39f24219a3851c5e0d136d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:46.267124 17139 consensus_queue.cc:260] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.267390 17139 ts_tablet_manager.cc:1428] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:11:46.267676 17485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab91c6406c24f48890affdde83502ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:46.267581 17338 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab91c6406c24f48890affdde83502ab (DEFAULT_TABLE table=test-table [id=81fe044a39f24219a3851c5e0d136d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:11:46.268523 17338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab91c6406c24f48890affdde83502ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:11:46.275563 17574 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Bootstrap starting.
I20260811 05:11:46.275729 17575 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Bootstrap starting.
I20260811 05:11:46.278475 17574 tablet_bootstrap.cc:654] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:46.278489 17575 tablet_bootstrap.cc:654] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Neither blocks nor log segments found. Creating new log.
I20260811 05:11:46.279379 17575 log.cc:824] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:46.279379 17574 log.cc:824] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:11:46.281324 17575 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: No bootstrap required, opened a new log
I20260811 05:11:46.281332 17574 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: No bootstrap required, opened a new log
I20260811 05:11:46.281513 17574 ts_tablet_manager.cc:1397] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:11:46.281513 17575 ts_tablet_manager.cc:1397] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:11:46.285311 17574 raft_consensus.cc:348] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.285311 17575 raft_consensus.cc:348] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.285583 17575 raft_consensus.cc:374] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:46.285583 17574 raft_consensus.cc:374] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:11:46.285643 17574 raft_consensus.cc:729] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f904223bb94df9838dc6a211b39e9f, State: Initialized, Role: FOLLOWER
I20260811 05:11:46.285643 17575 raft_consensus.cc:729] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ef0e46e435453db101c155ca7800ea, State: Initialized, Role: FOLLOWER
I20260811 05:11:46.286119 17574 consensus_queue.cc:260] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.286119 17575 consensus_queue.cc:260] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.286792 17411 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
I20260811 05:11:46.287146 17575 ts_tablet_manager.cc:1428] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:11:46.287182 17574 ts_tablet_manager.cc:1428] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:11:46.287638 17559 heartbeater.cc:503] Master 127.30.88.254:35195 was elected leader, sending a full tablet report...
W20260811 05:11:46.340042 17260 tablet.cc:2369] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:46.392618 17265 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:11:46.392823 17265 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.393985 17265 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:46.398046 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.394365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:57067 (local address 127.30.88.195:39735)
0811 05:11:46.394509 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.394514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.394808 (+ 294us) server_negotiation.cc:407] Connection header received
0811 05:11:46.394953 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.394956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.394996 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.395082 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:46.395377 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.395996 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397035 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.397482 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397622 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.397671 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.397874 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.397936 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:11:46.398052 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.394392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:39229 (local address 127.30.88.194:35755)
0811 05:11:46.394519 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.394526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.394721 (+ 195us) server_negotiation.cc:407] Connection header received
0811 05:11:46.394929 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.394935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.394999 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.395084 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:11:46.395382 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.395895 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397011 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.397464 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397584 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.397638 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.397844 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.397913 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:11:46.398075 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.394216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:35755 (local address 127.30.88.193:39229)
0811 05:11:46.394590 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.394602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.394758 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.395118 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.395121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.395150 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:46.395270 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.395275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.396006 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.396010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.396875 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.396885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.397586 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.397673 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.397725 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.397796 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.397894 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":6,"thread_start_us":132,"threads_started":1}
I20260811 05:11:46.398087 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.394218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39735 (local address 127.30.88.193:57067)
0811 05:11:46.394696 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.394724 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 05:11:46.394828 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:11:46.395088 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:11:46.395090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:11:46.395106 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:11:46.395268 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.395273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.396107 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.396110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.396895 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:11:46.396904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.397628 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.397632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:11:46.397725 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.397751 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:11:46.397842 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:11:46.397913 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":52,"spinlock_wait_cycles":24832,"thread_start_us":101,"threads_started":1}
I20260811 05:11:46.398727 17359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab91c6406c24f48890affdde83502ab" candidate_uuid: "053dc20c44024980bb590b2aeb68baa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ef0e46e435453db101c155ca7800ea" is_pre_election: true
I20260811 05:11:46.399008 17509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab91c6406c24f48890affdde83502ab" candidate_uuid: "053dc20c44024980bb590b2aeb68baa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f904223bb94df9838dc6a211b39e9f" is_pre_election: true
I20260811 05:11:46.399039 17359 raft_consensus.cc:2454] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 053dc20c44024980bb590b2aeb68baa9 in term 0.
I20260811 05:11:46.399283 17509 raft_consensus.cc:2454] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 053dc20c44024980bb590b2aeb68baa9 in term 0.
I20260811 05:11:46.399541 17136 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9, a0ef0e46e435453db101c155ca7800ea; no voters:
I20260811 05:11:46.399837 17265 raft_consensus.cc:2781] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:11:46.399932 17265 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:11:46.400004 17265 raft_consensus.cc:3037] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:46.401000 17265 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.401402 17265 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 1 election: Requested vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
I20260811 05:11:46.401746 17509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab91c6406c24f48890affdde83502ab" candidate_uuid: "053dc20c44024980bb590b2aeb68baa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0f904223bb94df9838dc6a211b39e9f"
I20260811 05:11:46.401867 17509 raft_consensus.cc:3037] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:46.401875 17359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab91c6406c24f48890affdde83502ab" candidate_uuid: "053dc20c44024980bb590b2aeb68baa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ef0e46e435453db101c155ca7800ea"
I20260811 05:11:46.401998 17359 raft_consensus.cc:3037] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:11:46.403076 17509 raft_consensus.cc:2454] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053dc20c44024980bb590b2aeb68baa9 in term 1.
I20260811 05:11:46.403231 17359 raft_consensus.cc:2454] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053dc20c44024980bb590b2aeb68baa9 in term 1.
I20260811 05:11:46.403414 17133 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9, a0f904223bb94df9838dc6a211b39e9f; no voters:
I20260811 05:11:46.403592 17265 raft_consensus.cc:2781] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:11:46.403785 17265 raft_consensus.cc:686] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 LEADER]: Becoming Leader. State: Replica: 053dc20c44024980bb590b2aeb68baa9, State: Running, Role: LEADER
I20260811 05:11:46.404026 17265 consensus_queue.cc:237] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:11:46.406776 17052 catalog_manager.cc:5495] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193). New cstate: current_term: 1 leader_uuid: "053dc20c44024980bb590b2aeb68baa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } health_report { overall_health: HEALTHY } } }
I20260811 05:11:46.451380 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.447745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.30.88.193:39929)
0811 05:11:46.448044 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 05:11:46.448053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:11:46.448076 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:11:46.448138 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:11:46.448140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:11:46.448186 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:11:46.448310 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:11:46.448568 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.449056 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.449849 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:11:46.450101 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:11:46.450241 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:11:46.450518 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:11:46.450930 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:11:46.451063 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:11:46.451123 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:11:46.451183 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260811 05:11:46.455161 17359 raft_consensus.cc:1261] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Refusing update from remote peer 053dc20c44024980bb590b2aeb68baa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:46.455161 17509 raft_consensus.cc:1261] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Refusing update from remote peer 053dc20c44024980bb590b2aeb68baa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:11:46.455844 17584 consensus_queue.cc:1035] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:11:46.456008 17265 consensus_queue.cc:1035] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:11:46.457896 17560 tablet.cc:2369] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:11:46.463097 17591 mvcc.cc:205] Tried to move back new op lower bound from 7317197236033867776 to 7317197235833430016. Current Snapshot: MvccSnapshot[applied={T|T < 7317197236033867776}]
I20260811 05:11:46.471238 17049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:46.471410 17049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:11:46.473646 17049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Sending DeleteTablet for 3 replicas of tablet 1ab91c6406c24f48890affdde83502ab
I20260811 05:11:46.474321 17485 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab91c6406c24f48890affdde83502ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:46 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:40584
I20260811 05:11:46.474466 17185 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab91c6406c24f48890affdde83502ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:46 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60012
I20260811 05:11:46.474522 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17093
I20260811 05:11:46.474794 17338 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab91c6406c24f48890affdde83502ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:11:46 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:33984
I20260811 05:11:46.476423 17603 tablet_replica.cc:330] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: stopping tablet replica
I20260811 05:11:46.476701 17603 raft_consensus.cc:2229] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:46.476941 17603 raft_consensus.cc:2258] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:46.478672 17601 tablet_replica.cc:330] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: stopping tablet replica
I20260811 05:11:46.478982 17601 raft_consensus.cc:2229] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:11:46.479035 17603 ts_tablet_manager.cc:1905] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:46.479298 17601 raft_consensus.cc:2258] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:11:46.480650 17601 ts_tablet_manager.cc:1905] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:11:46.481740 17603 ts_tablet_manager.cc:1918] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:46.481840 17603 log.cc:1196] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/1ab91c6406c24f48890affdde83502ab
I20260811 05:11:46.482178 17603 ts_tablet_manager.cc:1939] T 1ab91c6406c24f48890affdde83502ab P a0ef0e46e435453db101c155ca7800ea: Deleting consensus metadata
I20260811 05:11:46.483193 17601 ts_tablet_manager.cc:1918] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:11:46.483292 17601 log.cc:1196] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/1ab91c6406c24f48890affdde83502ab
I20260811 05:11:46.483335 17035 catalog_manager.cc:4841] TS a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): tablet 1ab91c6406c24f48890affdde83502ab (table test-table [id=81fe044a39f24219a3851c5e0d136d1c]) successfully deleted
I20260811 05:11:46.483645 17601 ts_tablet_manager.cc:1939] T 1ab91c6406c24f48890affdde83502ab P a0f904223bb94df9838dc6a211b39e9f: Deleting consensus metadata
I20260811 05:11:46.484555 17033 catalog_manager.cc:4841] TS a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): tablet 1ab91c6406c24f48890affdde83502ab (table test-table [id=81fe044a39f24219a3851c5e0d136d1c]) successfully deleted
W20260811 05:11:46.485406 17032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:39929 (error 108)
I20260811 05:11:46.485668 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17264
I20260811 05:11:46.491890 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:11:46.491611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:39929 (local address 127.0.0.1:60026)
0811 05:11:46.491729 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:11:46.491801 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:39929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:11:46.492053 17032 catalog_manager.cc:4584] TS 053dc20c44024980bb590b2aeb68baa9 (127.30.88.193:39929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ab91c6406c24f48890affdde83502ab: Network error: Client connection negotiation failed: client connection to 127.30.88.193:39929: connect: Connection refused (error 111)
I20260811 05:11:46.495481 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17415
I20260811 05:11:46.502368 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17012
I20260811 05:12:06.511847 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--webserver_interface=127.30.88.254
--webserver_port=35015
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:06.623783 17605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:06.624044 17605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:06.624117 17605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:06.628064 17605 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:12:06.628115 17605 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:12:06.628130 17605 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:12:06.628144 17605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:06.628189 17605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:06.628223 17605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:06.628244 17605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:06.628275 17605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:06.633523 17605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=35015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:06.634543 17605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:06.635557 17605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:06.643800 17613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:06.643784 17610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:06.643772 17611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:06.644170 17605 server_base.cc:1034] running on GCE node
I20260811 05:12:06.644678 17605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:06.645663 17605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:06.646874 17605 hybrid_clock.cc:648] HybridClock initialized: now 1786425126646818 us; error 36 us; skew 500 ppm
Aug 11 05:12:06 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425126, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:06.653605 17605 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:06.654456 17605 webserver.cc:460] Webserver started at http://127.30.88.254:35015/ using document root <none> and password file <none>
I20260811 05:12:06.655072 17605 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:06.655139 17605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:06.658783 17605 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:06.662665 17622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:06.663878 17605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:06.664011 17605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "2f43a28146084c6a8722ce404097e29c"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "84b050e19b041a95d80b31a936127954"
server_key_iv: "2d9e95e24f503ae9d4eb8c5a06b8fa6b"
server_key_version: "encryptionkey@0"
I20260811 05:12:06.664532 17605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:06.690074 17605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:06.693104 17605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:06.693289 17605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:06.700323 17605 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35195
I20260811 05:12:06.700613 17680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35195 every 8 connection(s)
I20260811 05:12:06.701351 17605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:06.705426 17681 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:12:06.707046 17681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap starting.
I20260811 05:12:06.709458 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17605
I20260811 05:12:06.709874 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:39929
--local_ip_for_outbound_sockets=127.30.88.193
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=34309
--webserver_interface=127.30.88.193
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:12:06.723840 17681 log.cc:824] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:06.734633 17681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:12:06.735051 17681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Bootstrap complete.
I20260811 05:12:06.738886 17681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:12:06.739283 17681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Initialized, Role: FOLLOWER
I20260811 05:12:06.739780 17681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:12:06.739918 17681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:06.739996 17681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:06.740104 17681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:12:06.741914 17681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:12:06.742220 17681 leader_election.cc:302] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f43a28146084c6a8722ce404097e29c; no voters:
I20260811 05:12:06.742507 17681 leader_election.cc:288] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:12:06.742630 17686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:12:06.742879 17686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [term 3 LEADER]: Becoming Leader. State: Replica: 2f43a28146084c6a8722ce404097e29c, State: Running, Role: LEADER
I20260811 05:12:06.743234 17686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } }
I20260811 05:12:06.743613 17681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:06.744660 17688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:12:06.744778 17688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:06.745434 17696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:06.745465 17689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f43a28146084c6a8722ce404097e29c. Latest consensus state: current_term: 3 leader_uuid: "2f43a28146084c6a8722ce404097e29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f43a28146084c6a8722ce404097e29c" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35195 } } }
I20260811 05:12:06.745612 17689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:06.749523 17696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:06.750043 17696 catalog_manager.cc:1259] Loaded cluster ID: 4bfe84061c4b42fd9db71970e3050139
I20260811 05:12:06.750134 17696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:06.751945 17696 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:06.752573 17696 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Loaded TSK: 0
I20260811 05:12:06.752656 17696 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f43a28146084c6a8722ce404097e29c: Loaded TSK: 1
F20260811 05:12:06.753269 17696 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4747cad9ad google::LogMessage::Fail() at ??:0
@ 0x7f4747cb088a google::LogMessage::SendToLog() at ??:0
@ 0x7f4747cad4de google::LogMessage::Flush() at ??:0
@ 0x7f4747caea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f474a97118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f474a969f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f47526127bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f47525ec99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f47525ed178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f475262edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f475264d82b std::function<>::operator()() at ??:0
@ 0x7f47525ecb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f47525edf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f47525ea579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f475262e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4749eff3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4749effc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4749f013f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4749ef0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f474b5346db start_thread at ??:0
@ 0x7f47467ab61f clone at ??:0
W20260811 05:12:06.839396 17684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:06.839644 17684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:06.839694 17684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:06.843847 17684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:06.843899 17684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:06.843955 17684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:06.849386 17684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:39929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=34309
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:06.850327 17684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:06.851282 17684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:06.860450 17735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:06.860591 17738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:06.860450 17736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:06.860752 17684 server_base.cc:1034] running on GCE node
I20260811 05:12:06.861223 17684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:06.861786 17684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:06.862913 17684 hybrid_clock.cc:648] HybridClock initialized: now 1786425126862877 us; error 48 us; skew 500 ppm
Aug 11 05:12:06 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425126, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:06.865479 17684 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:12:06.866308 17684 webserver.cc:460] Webserver started at http://127.30.88.193:34309/ using document root <none> and password file <none>
I20260811 05:12:06.866892 17684 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:06.866990 17684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:06.870520 17684 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:06.872694 17746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:06.873867 17684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:12:06.873993 17684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "053dc20c44024980bb590b2aeb68baa9"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "cdfe8614d2bcb9971a7fee52ebe5d665"
server_key_iv: "473fbaeadd27bb790f8c418f26af09b6"
server_key_version: "encryptionkey@0"
I20260811 05:12:06.874482 17684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:06.907146 17684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:06.910315 17684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:06.910516 17684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:06.911113 17684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:06.912235 17753 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:12:06.917587 17684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:12:06.917652 17684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 05:12:06.917725 17684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:12:06.919377 17684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:12:06.919441 17684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:06.919727 17753 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Bootstrap starting.
I20260811 05:12:06.929744 17684 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:39929
I20260811 05:12:06.929951 17870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:39929 every 8 connection(s)
I20260811 05:12:06.930732 17684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:06.937183 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17684
I20260811 05:12:06.937637 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:35755
--local_ip_for_outbound_sockets=127.30.88.194
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=44913
--webserver_interface=127.30.88.194
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:12:06.941020 17753 log.cc:824] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:06.950788 17753 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:12:06.951297 17753 tablet_bootstrap.cc:492] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Bootstrap complete.
I20260811 05:12:06.952702 17753 ts_tablet_manager.cc:1397] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.012s
I20260811 05:12:06.955770 17753 raft_consensus.cc:348] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:06.956288 17753 raft_consensus.cc:729] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053dc20c44024980bb590b2aeb68baa9, State: Initialized, Role: FOLLOWER
I20260811 05:12:06.956787 17753 consensus_queue.cc:260] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:06.957773 17753 ts_tablet_manager.cc:1428] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
W20260811 05:12:06.986935 17874 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:12:06.932584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:58345)
0811 05:12:06.933035 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:06.933086 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:06.933846 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:06.985792 (+ 51946us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":130,"threads_started":1}
W20260811 05:12:06.987553 17871 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:35195 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:35195: Client connection negotiation failed: client connection to 127.30.88.254:35195: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:12:06.988258 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:06.987997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:38715)
0811 05:12:06.988124 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:06.988194 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 05:12:06.988907 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:06.988705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:35543)
0811 05:12:06.988806 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:06.988854 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 05:12:06.989063 17871 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:12:07.054774 17875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:07.055097 17875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:07.055140 17875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:07.059285 17875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:07.059337 17875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:07.059392 17875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:07.064262 17875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:35755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=44913
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:07.065201 17875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:07.066147 17875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:12:07.075673 17875 server_base.cc:1034] running on GCE node
W20260811 05:12:07.075628 17883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:07.075656 17885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:07.075628 17888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:07.076270 17875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:07.076848 17875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:07.078055 17875 hybrid_clock.cc:648] HybridClock initialized: now 1786425127078022 us; error 41 us; skew 500 ppm
Aug 11 05:12:07 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425127, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:07.080773 17875 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:12:07.081648 17875 webserver.cc:460] Webserver started at http://127.30.88.194:44913/ using document root <none> and password file <none>
I20260811 05:12:07.082191 17875 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:07.082278 17875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:07.085955 17875 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:07.087975 17897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.089241 17875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:07.089358 17875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "a0ef0e46e435453db101c155ca7800ea"
format_stamp: "Formatted at 2026-08-11 05:11:24 on dist-test-slave-7c35"
server_key: "e12e49592868995c99cd5f41abeb58a1"
server_key_iv: "623266fcb597f01cf40282105fc5a244"
server_key_version: "encryptionkey@0"
I20260811 05:12:07.090045 17875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:07.107260 17875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:07.110478 17875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:07.110677 17875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:07.111299 17875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:07.112109 17875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:07.112187 17875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.112257 17875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:07.112303 17875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.121011 17875 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:35755
I20260811 05:12:07.121059 18027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:35755 every 8 connection(s)
I20260811 05:12:07.121960 17875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:07.125378 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.123838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:55335)
0811 05:12:07.124292 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.124699 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":133,"threads_started":1}
W20260811 05:12:07.125802 18028 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:35195 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:35195: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
I20260811 05:12:07.126248 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 17875
I20260811 05:12:07.126410 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.126156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:52067)
0811 05:12:07.126281 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.126345 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 05:12:07.126822 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:39735
--local_ip_for_outbound_sockets=127.30.88.195
--tserver_master_addrs=127.30.88.254:35195
--webserver_port=35205
--webserver_interface=127.30.88.195
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:12:07.127094 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.126825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:57681)
0811 05:12:07.126981 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.127039 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 05:12:07.127310 18028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:12:07.237260 18032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:07.237478 18032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:07.237517 18032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:07.241173 18032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:07.241211 18032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:07.241276 18032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:07.246042 18032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:39735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=35205
--tserver_master_addrs=127.30.88.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:07.246944 18032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:07.247783 18032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:07.256727 18042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:07.256775 18039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:07.256824 18032 server_base.cc:1034] running on GCE node
W20260811 05:12:07.256727 18040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:07.257498 18032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:07.258071 18032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:07.259236 18032 hybrid_clock.cc:648] HybridClock initialized: now 1786425127259212 us; error 36 us; skew 500 ppm
Aug 11 05:12:07 dist-test-slave-7c35 krb5kdc[16404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425127, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:07.261801 18032 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:12:07.262602 18032 webserver.cc:460] Webserver started at http://127.30.88.195:35205/ using document root <none> and password file <none>
I20260811 05:12:07.263191 18032 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:07.263258 18032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:07.266773 18032 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:07.268690 18049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.269757 18032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:12:07.269871 18032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "a0f904223bb94df9838dc6a211b39e9f"
format_stamp: "Formatted at 2026-08-11 05:11:25 on dist-test-slave-7c35"
server_key: "d80c79d199afb8bbadc3b6adac7b0a37"
server_key_iv: "46295a0333b7347b1e2290c1c3bc0b0e"
server_key_version: "encryptionkey@0"
I20260811 05:12:07.270380 18032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:07.288627 18032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:07.291311 18032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:07.291502 18032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:07.292060 18032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:07.292869 18032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:07.292918 18032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.292989 18032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:07.293021 18032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:07.301443 18032 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:39735
I20260811 05:12:07.301477 18179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:39735 every 8 connection(s)
I20260811 05:12:07.302289 18032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:07.303833 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18032
I20260811 05:12:07.306026 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.304343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:46869)
0811 05:12:07.304787 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.305361 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":141,"threads_started":1}
W20260811 05:12:07.306443 18180 heartbeater.cc:644] Failed to heartbeat to 127.30.88.254:35195 (0 consecutive failures): Network error: Failed to ping master at 127.30.88.254:35195: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
I20260811 05:12:07.306975 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.306745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:46351)
0811 05:12:07.306870 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.306925 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 05:12:07.307466 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.307284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:54357)
0811 05:12:07.307374 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.307415 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 05:12:07.307650 18180 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:12:07.990087 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:07.989647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:39875)
0811 05:12:07.989889 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:07.989942 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260811 05:12:08.128216 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.127820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:57333)
0811 05:12:08.128068 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:08.128120 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
I20260811 05:12:08.252548 18186 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:08.252755 18186 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:08.253872 18186 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
Aug 11 05:12:08 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425126, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.194@KRBTEST.COM
Aug 11 05:12:08 dist-test-slave-7c35 krb5kdc[16404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425126, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.195@KRBTEST.COM
I20260811 05:12:08.267143 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.254046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:39735 (local address 127.30.88.193:42633)
0811 05:12:08.254156 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:08.254173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:08.254278 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:08.256070 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:08.256076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:08.256613 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:08.257088 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:08.257102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.258341 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.258343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:08.259361 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:08.259373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.260043 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.260047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:08.260301 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:08.260935 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:08.260954 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:08.262977 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:08.264834 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:08.264836 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:08.264838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:08.265125 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:08.265492 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:08.265497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:08.265616 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:08.266196 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:08.266202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:08.266433 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 05:12:08.266613 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 05:12:08.266952 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":141,"spinlock_wait_cycles":10496}
I20260811 05:12:08.267177 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.254392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:58235 (local address 127.30.88.194:35755)
0811 05:12:08.254714 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:08.254731 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:08.254760 (+ 29us) server_negotiation.cc:407] Connection header received
0811 05:12:08.255421 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:08.255427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:08.255719 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:08.256021 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:08.257273 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.258179 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.259474 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.259922 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.260100 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:08.262833 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:08.262839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:08.262853 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:08.262903 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:08.264622 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265217 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:08.265224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:08.265228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:08.265370 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265703 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:08.265708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:08.265713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:08.266075 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:08.266211 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:08.266724 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 05:12:08.267018 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":83,"threads_started":1}
I20260811 05:12:08.267225 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.254223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:42633 (local address 127.30.88.195:39735)
0811 05:12:08.254645 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:08.254656 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:08.254691 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:08.255383 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:08.255389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:08.255718 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:08.256023 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:08.257271 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.258202 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.259478 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.259921 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.260099 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:08.263130 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:08.263134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:08.263143 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:08.263166 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:08.264731 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265245 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:08.265249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:08.265252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:08.265378 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265729 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:08.265731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:08.265732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:08.266072 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:08.266214 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:08.266721 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 05:12:08.267040 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":77,"threads_started":1}
I20260811 05:12:08.267263 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.254097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:35755 (local address 127.30.88.193:58235)
0811 05:12:08.254384 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:08.254397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:08.254512 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:08.256069 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:08.256075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:08.256613 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:08.257088 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:08.257102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.258314 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.258317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:08.259357 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:08.259367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:08.260044 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:08.260047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:08.260352 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:08.260935 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:08.260962 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:08.262610 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:08.264731 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:08.264736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:08.264748 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:08.265108 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:08.265465 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:08.265471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:08.265474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:08.265610 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:08.266195 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:08.266201 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:08.266437 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 05:12:08.266613 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 05:12:08.267040 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":518,"spinlock_wait_cycles":30720,"thread_start_us":57,"threads_started":1}
W20260811 05:12:08.270308 17748 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
W20260811 05:12:08.270886 17750 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
I20260811 05:12:08.271013 17750 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9; no voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f
I20260811 05:12:08.271252 18186 raft_consensus.cc:2726] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:12:08.308452 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.308075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:38367)
0811 05:12:08.308310 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:08.308359 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260811 05:12:08.991238 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:08.990787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:44975)
0811 05:12:08.991088 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:08.991141 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":140,"threads_started":1}
I20260811 05:12:09.129295 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:09.128885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:37513)
0811 05:12:09.129156 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:09.129212 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":70,"threads_started":1}
I20260811 05:12:09.309502 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:09.309088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:53907)
0811 05:12:09.309338 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:09.309395 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":68,"threads_started":1}
I20260811 05:12:09.963192 18196 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:09.963382 18196 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:09.964046 18196 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
W20260811 05:12:09.964689 17748 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
W20260811 05:12:09.964772 17750 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
I20260811 05:12:09.964895 17750 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9; no voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f
I20260811 05:12:09.965044 18196 raft_consensus.cc:2726] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:12:09.992350 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:09.991896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:56345)
0811 05:12:09.992188 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:09.992250 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":98,"threads_started":1}
I20260811 05:12:10.130522 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:10.130119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:59881)
0811 05:12:10.130357 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:10.130411 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260811 05:12:10.310662 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:10.310289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:47467)
0811 05:12:10.310525 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:10.310576 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":81,"threads_started":1}
I20260811 05:12:10.993561 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:10.993103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:60359)
0811 05:12:10.993405 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:10.993459 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":98,"threads_started":1}
I20260811 05:12:11.131603 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:11.131198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:58951)
0811 05:12:11.131451 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:11.131506 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":68,"threads_started":1}
I20260811 05:12:11.311872 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:11.311492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:49329)
0811 05:12:11.311732 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:11.311784 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260811 05:12:11.486493 18203 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:11.486618 18203 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:11.487106 18203 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
W20260811 05:12:11.487854 17748 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
W20260811 05:12:11.487999 17750 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
I20260811 05:12:11.488073 17750 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9; no voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f
I20260811 05:12:11.488207 18203 raft_consensus.cc:2726] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:12:11.995024 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:11.994513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:37091)
0811 05:12:11.994851 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:11.994925 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260811 05:12:12.132797 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:12.132398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:46977)
0811 05:12:12.132651 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:12.132708 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260811 05:12:12.312996 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:12.312592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:42241)
0811 05:12:12.312852 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:12.312905 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260811 05:12:12.996186 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:12.995766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:50591)
0811 05:12:12.996032 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:12.996087 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":75,"threads_started":1}
I20260811 05:12:13.133930 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:13.133551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:34447)
0811 05:12:13.133790 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:13.133842 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":69,"threads_started":1}
I20260811 05:12:13.314208 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:13.313844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:34677)
0811 05:12:13.314075 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:13.314126 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260811 05:12:13.997310 18211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:13.996912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:36903)
0811 05:12:13.997160 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:13.997215 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260811 05:12:14.135152 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:14.134668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:35593)
0811 05:12:14.135018 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:14.135069 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":71,"threads_started":1}
I20260811 05:12:14.315430 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:14.315014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:50301)
0811 05:12:14.315290 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:14.315344 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":71,"threads_started":1}
I20260811 05:12:14.601580 18215 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:14.601704 18215 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:14.602082 18215 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
W20260811 05:12:14.603019 17750 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
W20260811 05:12:14.603210 17748 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
I20260811 05:12:14.603310 17748 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9; no voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f
I20260811 05:12:14.603513 18215 raft_consensus.cc:2726] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:12:14.998377 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:14.998006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:59911)
0811 05:12:14.998234 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:14.998284 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260811 05:12:15.136276 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:15.135846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:59099)
0811 05:12:15.136126 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:15.136183 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
I20260811 05:12:15.316524 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:15.316113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:36645)
0811 05:12:15.316362 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:15.316436 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260811 05:12:15.999547 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:15.999135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:34809)
0811 05:12:15.999398 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:15.999453 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260811 05:12:16.137365 18220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:16.136969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:54733)
0811 05:12:16.137220 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:16.137282 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260811 05:12:16.317560 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:16.317177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:37367)
0811 05:12:16.317412 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:16.317461 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260811 05:12:17.000717 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:17.000344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:44665)
0811 05:12:17.000577 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:17.000625 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
I20260811 05:12:17.138482 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:17.138096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:40733)
0811 05:12:17.138335 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:17.138389 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260811 05:12:17.318717 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:17.318353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:45391)
0811 05:12:17.318581 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:17.318632 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260811 05:12:18.001766 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:18.001388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:46065)
0811 05:12:18.001618 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:18.001670 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260811 05:12:18.120005 18226 raft_consensus.cc:482] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:18.120136 18226 raft_consensus.cc:504] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0f904223bb94df9838dc6a211b39e9f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 39735 } } peers { permanent_uuid: "a0ef0e46e435453db101c155ca7800ea" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 35755 } } peers { permanent_uuid: "053dc20c44024980bb590b2aeb68baa9" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 39929 } }
I20260811 05:12:18.120600 18226 leader_election.cc:288] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735), a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755)
W20260811 05:12:18.121383 17750 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ef0e46e435453db101c155ca7800ea (127.30.88.194:35755): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
W20260811 05:12:18.121475 17748 leader_election.cc:341] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0f904223bb94df9838dc6a211b39e9f (127.30.88.195:39735): Not found: Tablet not found: 1ab91c6406c24f48890affdde83502ab
I20260811 05:12:18.121582 17748 leader_election.cc:302] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 053dc20c44024980bb590b2aeb68baa9; no voters: a0ef0e46e435453db101c155ca7800ea, a0f904223bb94df9838dc6a211b39e9f
I20260811 05:12:18.121764 18226 raft_consensus.cc:2726] T 1ab91c6406c24f48890affdde83502ab P 053dc20c44024980bb590b2aeb68baa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:12:18.139570 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:18.139195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:53949)
0811 05:12:18.139424 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:18.139485 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":74,"threads_started":1}
I20260811 05:12:18.319808 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:18.319389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:36953)
0811 05:12:18.319669 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:18.319720 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260811 05:12:19.003085 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:19.002582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:39031)
0811 05:12:19.002913 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:19.002979 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260811 05:12:19.140673 18231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:19.140283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:55977)
0811 05:12:19.140530 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:19.140585 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260811 05:12:19.320892 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:19.320531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:47565)
0811 05:12:19.320754 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:19.320805 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260811 05:12:20.004295 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:20.003834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:37779)
0811 05:12:20.004139 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:20.004196 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":73,"threads_started":1}
I20260811 05:12:20.141747 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:20.141364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:33027)
0811 05:12:20.141603 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:20.141656 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
I20260811 05:12:20.322079 18235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:20.321673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:36959)
0811 05:12:20.321935 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:20.321989 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
I20260811 05:12:21.005460 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:21.005014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:34299)
0811 05:12:21.005260 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:21.005312 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":87,"threads_started":1}
I20260811 05:12:21.142803 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:21.142420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:54237)
0811 05:12:21.142662 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:21.142713 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
I20260811 05:12:21.323213 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:21.322795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:48303)
0811 05:12:21.323071 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:21.323124 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260811 05:12:22.007012 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:22.006544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.193:43023)
0811 05:12:22.006807 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:22.006897 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260811 05:12:22.143968 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:22.143606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.194:58943)
0811 05:12:22.143836 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:22.143884 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260811 05:12:22.310120 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17684
I20260811 05:12:22.321449 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 17875
I20260811 05:12:22.324747 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:22.324314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35195 (local address 127.30.88.195:33277)
0811 05:12:22.324607 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:22.324657 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.254:35195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
I20260811 05:12:22.329094 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18032
2026-08-11T05:12:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59257 ms)
[----------] 30 tests from SecurityITest (240186 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:22 dist-test-slave-7c35 krb5kdc[18246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:22 dist-test-slave-7c35 krb5kdc[18246](info): set up 2 sockets
Aug 11 05:12:22 dist-test-slave-7c35 krb5kdc[18246](info): commencing operation
krb5kdc: starting...
W20260811 05:12:23.435360 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:23 dist-test-slave-7c35 krb5kdc[18246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:23.558794 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34951
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:23.678408 18260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:23.678755 18260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:23.678821 18260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:23.682768 18260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:23.682824 18260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:23.682894 18260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:23.682925 18260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:23.682957 18260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:23.688095 18260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:34951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:23.689148 18260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:23.690161 18260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:12:23.698948 18260 server_base.cc:1034] running on GCE node
W20260811 05:12:23.698880 18266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:23.698937 18268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:23.698877 18270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:23.699733 18260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:23.700829 18260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:23.702003 18260 hybrid_clock.cc:648] HybridClock initialized: now 1786425143701983 us; error 30 us; skew 500 ppm
Aug 11 05:12:23 dist-test-slave-7c35 krb5kdc[18246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:23.704818 18260 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:23.705641 18260 webserver.cc:460] Webserver started at http://127.30.88.254:40019/ using document root <none> and password file <none>
I20260811 05:12:23.706212 18260 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:23.706274 18260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:23.706429 18260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:23.708380 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "e340a58ea88746d4bf5a6a913df0979b"
format_stamp: "Formatted at 2026-08-11 05:12:23 on dist-test-slave-7c35"
server_key: "c8ea83eae446093ad90978df74f7bb64"
server_key_iv: "bc19a171bdd32b3e25ffa0dc87b36c80"
server_key_version: "encryptionkey@0"
I20260811 05:12:23.708889 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "e340a58ea88746d4bf5a6a913df0979b"
format_stamp: "Formatted at 2026-08-11 05:12:23 on dist-test-slave-7c35"
server_key: "c8ea83eae446093ad90978df74f7bb64"
server_key_iv: "bc19a171bdd32b3e25ffa0dc87b36c80"
server_key_version: "encryptionkey@0"
I20260811 05:12:23.712323 18260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:23.714619 18278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:23.715898 18260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:23.716023 18260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "e340a58ea88746d4bf5a6a913df0979b"
format_stamp: "Formatted at 2026-08-11 05:12:23 on dist-test-slave-7c35"
server_key: "c8ea83eae446093ad90978df74f7bb64"
server_key_iv: "bc19a171bdd32b3e25ffa0dc87b36c80"
server_key_version: "encryptionkey@0"
I20260811 05:12:23.716123 18260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:23.745301 18260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:23.748252 18260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:23.748485 18260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:23.756086 18260 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:34951
I20260811 05:12:23.756114 18336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:34951 every 8 connection(s)
I20260811 05:12:23.757025 18260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:23.759826 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:23.765045 18337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: Bootstrap starting.
I20260811 05:12:23.766192 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18260
I20260811 05:12:23.766353 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:23.766710 31075 external_mini_cluster.cc:1428] Setting key e2c0a9c0ce6c2310f32352f55edd914e
I20260811 05:12:23.767766 18337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:23.768610 18337 log.cc:824] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:23.770928 18337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: No bootstrap required, opened a new log
I20260811 05:12:23.775283 18337 raft_consensus.cc:348] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } }
I20260811 05:12:23.775530 18337 raft_consensus.cc:374] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:23.775609 18337 raft_consensus.cc:729] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e340a58ea88746d4bf5a6a913df0979b, State: Initialized, Role: FOLLOWER
I20260811 05:12:23.776054 18337 consensus_queue.cc:260] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } }
I20260811 05:12:23.776176 18337 raft_consensus.cc:388] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:23.776270 18337 raft_consensus.cc:482] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:23.776358 18337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 0 FOLLOWER]: Advancing to term 1
Aug 11 05:12:23 dist-test-slave-7c35 krb5kdc[18246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:23.777258 18337 raft_consensus.cc:504] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } }
I20260811 05:12:23.777617 18337 leader_election.cc:302] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e340a58ea88746d4bf5a6a913df0979b; no voters:
I20260811 05:12:23.777957 18337 leader_election.cc:288] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:23.778100 18343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:23.778373 18343 raft_consensus.cc:686] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [term 1 LEADER]: Becoming Leader. State: Replica: e340a58ea88746d4bf5a6a913df0979b, State: Running, Role: LEADER
I20260811 05:12:23.778728 18343 consensus_queue.cc:237] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } }
I20260811 05:12:23.779318 18337 sys_catalog.cc:564] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:23.780591 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e340a58ea88746d4bf5a6a913df0979b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } } }
I20260811 05:12:23.780752 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:23.781070 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e340a58ea88746d4bf5a6a913df0979b. Latest consensus state: current_term: 1 leader_uuid: "e340a58ea88746d4bf5a6a913df0979b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e340a58ea88746d4bf5a6a913df0979b" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 34951 } } }
I20260811 05:12:23.781188 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:23.784443 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:23.767995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.30.88.254:34951)
0811 05:12:23.768540 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:23.768558 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:23.768598 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:12:23.769435 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:23.769463 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:23.769849 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:23.770253 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:23.771218 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:23.772735 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:23.773719 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:23.774464 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:23.774622 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:23.777904 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:23.777912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:23.777929 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:23.777961 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:23.780391 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:23.781089 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:23.781097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:23.781104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:23.781272 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:23.781983 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:23.781987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:23.781988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:23.782573 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:23.782811 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:23.783481 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 05:12:23.783731 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":183,"threads_started":1}
W20260811 05:12:23.785158 18361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:12:23.785252 18361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:12:23.785369 18358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:23.786255 18358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:23.792289 18358 catalog_manager.cc:1347] Generated new cluster ID: 03e4ee03f8024fa4bff780cf3b69fed3
I20260811 05:12:23.792366 18358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:23.817149 18358 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:23.818176 18358 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:23.828889 18358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: Generated new TSK 0
I20260811 05:12:23.829552 18358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:12:23.864019 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:23.990483 18367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:23.990716 18367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:23.990756 18367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:23.994575 18367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:23.994623 18367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:23.994678 18367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:23.999634 18367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:24.000712 18367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:24.001633 18367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:24.010895 18375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.010912 18372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.010926 18373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:24.011300 18367 server_base.cc:1034] running on GCE node
I20260811 05:12:24.011695 18367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:24.012351 18367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:24.013586 18367 hybrid_clock.cc:648] HybridClock initialized: now 1786425144013558 us; error 39 us; skew 500 ppm
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:24.016605 18367 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:12:24.017417 18367 webserver.cc:460] Webserver started at http://127.30.88.193:37755/ using document root <none> and password file <none>
I20260811 05:12:24.017972 18367 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:24.018064 18367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:24.018291 18367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:24.020422 18367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "f25d702d825e4aacb7b4c7a6f7119eef"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "728b2de0500730b1a30d7402a4f27196"
server_key_iv: "ebe4b41f9abd864d5a549221b472e07f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.020996 18367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "f25d702d825e4aacb7b4c7a6f7119eef"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "728b2de0500730b1a30d7402a4f27196"
server_key_iv: "ebe4b41f9abd864d5a549221b472e07f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.024590 18367 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:24.027469 18384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.028704 18367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:24.028841 18367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "f25d702d825e4aacb7b4c7a6f7119eef"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "728b2de0500730b1a30d7402a4f27196"
server_key_iv: "ebe4b41f9abd864d5a549221b472e07f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.028965 18367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:24.049021 18367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:24.052191 18367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:24.052426 18367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:24.053061 18367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:24.053977 18367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:24.054049 18367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.054118 18367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:24.054190 18367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.063997 18367 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:44741
I20260811 05:12:24.064024 18515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:44741 every 8 connection(s)
I20260811 05:12:24.064962 18367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:24.071627 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18367
I20260811 05:12:24.071770 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:24.072088 31075 external_mini_cluster.cc:1428] Setting key 58a107ca7a2d1a9b89275e288ed85bbc
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:24.078986 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.066923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:51581 (local address 127.30.88.254:34951)
0811 05:12:24.067063 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.067067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.067897 (+ 830us) server_negotiation.cc:407] Connection header received
0811 05:12:24.069202 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.069205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.069260 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.069359 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:24.070500 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.071106 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.072431 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.072668 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.072781 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.075294 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:24.075298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:24.075300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:24.075325 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:24.076815 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.077358 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.077363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.077364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.077450 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.077797 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.077801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.077803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.077990 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:24.078101 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.078661 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.078788 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:12:24.079919 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.067282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34951 (local address 127.30.88.193:51581)
0811 05:12:24.067752 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.067793 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.068998 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.069527 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.069534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.069858 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:24.070344 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.070359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.071238 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.071243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.072309 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.072320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.072821 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.072826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.073014 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.073615 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:24.073632 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:24.075125 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:24.076949 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.076954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.076968 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.077251 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.077546 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.077549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.077551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.077695 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.078106 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:24.078114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:24.078429 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.078616 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.078944 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":123,"threads_started":1}
I20260811 05:12:24.081152 18516 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34951
I20260811 05:12:24.081409 18516 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:24.081892 18516 heartbeater.cc:511] Master 127.30.88.254:34951 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:12:24.083585 18295 ts_manager.cc:194] Registered new tserver with Master: f25d702d825e4aacb7b4c7a6f7119eef (127.30.88.193:44741)
I20260811 05:12:24.084755 18295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:51581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:12:24.101697 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:24.212815 18521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:24.213089 18521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:24.213164 18521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:24.217163 18521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:24.217243 18521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:24.217329 18521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:24.222743 18521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:24.223748 18521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:24.224769 18521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:24.235222 18530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.235224 18526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.235224 18528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:24.235260 18521 server_base.cc:1034] running on GCE node
I20260811 05:12:24.235854 18521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:24.236557 18521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:24.237778 18521 hybrid_clock.cc:648] HybridClock initialized: now 1786425144237750 us; error 45 us; skew 500 ppm
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:24.240878 18521 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:12:24.241786 18521 webserver.cc:460] Webserver started at http://127.30.88.194:43771/ using document root <none> and password file <none>
I20260811 05:12:24.242322 18521 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:24.242403 18521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:24.242581 18521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:24.244774 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "353dcfd6e261e4622d86524874eb5cae"
server_key_iv: "946c289ae8f86c442d50d85cf2c2dd7f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.245373 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "353dcfd6e261e4622d86524874eb5cae"
server_key_iv: "946c289ae8f86c442d50d85cf2c2dd7f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.249392 18521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:12:24.252274 18540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.253468 18521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:24.253614 18521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "353dcfd6e261e4622d86524874eb5cae"
server_key_iv: "946c289ae8f86c442d50d85cf2c2dd7f"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.253721 18521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:24.274063 18521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:24.277397 18521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:24.277602 18521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:24.278188 18521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:24.279091 18521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:24.279139 18521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.279206 18521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:24.279237 18521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.288511 18521 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:43561
I20260811 05:12:24.288538 18671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:43561 every 8 connection(s)
I20260811 05:12:24.289304 18521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:24.298079 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18521
I20260811 05:12:24.298210 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:24.298547 31075 external_mini_cluster.cc:1428] Setting key 1f17e5fcc84bce4807ac78625ec17684
I20260811 05:12:24.302891 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.291015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:50797 (local address 127.30.88.254:34951)
0811 05:12:24.291155 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.291160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.292020 (+ 860us) server_negotiation.cc:407] Connection header received
0811 05:12:24.293266 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.293269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.293313 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.293383 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:24.294510 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.295115 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.296333 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.296548 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.296623 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.299104 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:24.299110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:24.299115 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:24.299146 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:24.300667 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.301175 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.301179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.301181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.301265 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.301618 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.301621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.301623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.301799 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:24.301927 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.302476 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.302654 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:12:24.303540 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.291483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34951 (local address 127.30.88.194:50797)
0811 05:12:24.291878 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.291919 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.293031 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.293563 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.293570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.293882 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:24.294350 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.294372 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.295228 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.295231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.296198 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.296209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.296678 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.296680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.296816 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.297321 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:24.297336 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:24.298945 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:24.300789 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.300795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.300808 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.301069 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.301359 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.301363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.301365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.301511 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.301950 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:24.301959 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:24.302232 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.302406 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.302669 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":91,"threads_started":1}
I20260811 05:12:24.304661 18672 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34951
I20260811 05:12:24.304904 18672 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:24.305418 18672 heartbeater.cc:511] Master 127.30.88.254:34951 requested a full tablet report, sending...
I20260811 05:12:24.306453 18295 ts_manager.cc:194] Registered new tserver with Master: bedd0f9ddd694ac7b7a3aa9c28e2fa83 (127.30.88.194:43561)
I20260811 05:12:24.307106 18295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:50797
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:12:24.326236 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:24.436197 18679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:24.436429 18679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:24.436488 18679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:24.440402 18679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:24.440449 18679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:24.440536 18679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:24.445504 18679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:24.446470 18679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:24.447484 18679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:24.456892 18685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.456905 18684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.456900 18687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:24.457123 18679 server_base.cc:1034] running on GCE node
I20260811 05:12:24.457674 18679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:24.458333 18679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:24.459513 18679 hybrid_clock.cc:648] HybridClock initialized: now 1786425144459494 us; error 30 us; skew 500 ppm
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:24.462051 18679 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:12:24.463003 18679 webserver.cc:460] Webserver started at http://127.30.88.195:36121/ using document root <none> and password file <none>
I20260811 05:12:24.463531 18679 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:24.463614 18679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:24.463806 18679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:24.465674 18679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "0b287f8cd288f971605436a651880486"
server_key_iv: "9b1643fa18c7d2adf89fa4ad9e36f00d"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.466215 18679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "0b287f8cd288f971605436a651880486"
server_key_iv: "9b1643fa18c7d2adf89fa4ad9e36f00d"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.469862 18679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 05:12:24.472325 18696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.473441 18679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:24.473578 18679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "0b287f8cd288f971605436a651880486"
server_key_iv: "9b1643fa18c7d2adf89fa4ad9e36f00d"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.473687 18679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:24.485628 18679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:24.488380 18679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:24.488579 18679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:24.489141 18679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:24.489939 18679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:24.490007 18679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.490104 18679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:24.490168 18679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.499738 18679 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:37053
I20260811 05:12:24.499765 18827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:37053 every 8 connection(s)
I20260811 05:12:24.500622 18679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:24.502337 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18679
I20260811 05:12:24.502470 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:24.502782 31075 external_mini_cluster.cc:1428] Setting key 210255a6f8a2d35b4a7e1c8c7ba22eac
Aug 11 05:12:24 dist-test-slave-7c35 krb5kdc[18246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:24.513945 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.502409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:53065 (local address 127.30.88.254:34951)
0811 05:12:24.502569 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.502574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.503466 (+ 892us) server_negotiation.cc:407] Connection header received
0811 05:12:24.505065 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.505068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.505125 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.505200 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:24.506309 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.506973 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.508256 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.508483 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.508582 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.510931 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:24.510934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:24.510936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:24.510960 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:24.512193 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.512672 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.512675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.512676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.512746 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.513045 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.513047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.513048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.513210 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:24.513282 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.513729 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.513824 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 05:12:24.514734 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.502788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:34951 (local address 127.30.88.195:53065)
0811 05:12:24.503312 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.503354 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.504829 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.505370 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.505377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.505713 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:24.506164 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.506177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.507118 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.507122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.508123 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.508134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.508628 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.508632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.508796 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.509291 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:24.509306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:24.510737 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:24.512294 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.512298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.512308 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.512588 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.512821 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:24.512824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:24.512825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:24.512959 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:24.513295 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:24.513299 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:24.513531 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.513684 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.513943 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":116,"threads_started":1}
I20260811 05:12:24.516144 18828 heartbeater.cc:347] Connected to a master server at 127.30.88.254:34951
I20260811 05:12:24.516413 18828 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:24.516968 18828 heartbeater.cc:511] Master 127.30.88.254:34951 requested a full tablet report, sending...
I20260811 05:12:24.518003 18295 ts_manager.cc:194] Registered new tserver with Master: cc75ea34b4cf498da2911e7cc1ca5ac6 (127.30.88.195:37053)
I20260811 05:12:24.518587 18295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:53065
I20260811 05:12:24.528646 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:24.537815 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.530799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.30.88.254:34951)
0811 05:12:24.530977 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.530984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.531191 (+ 207us) server_negotiation.cc:407] Connection header received
0811 05:12:24.531332 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.531334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.531375 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.531436 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:24.532172 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.532732 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.533556 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.533821 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.533895 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.535213 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:24.535215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:24.535217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:24.535239 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:24.536391 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.536755 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.536757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.536758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.536824 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:24.537082 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:24.537084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:24.537085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:24.537294 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:24.537396 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.537598 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.537692 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 05:12:24.540699 18295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:24.543567 18295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:24.558290 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.551757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43561 (local address 127.0.0.1:34660)
0811 05:12:24.552135 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.552185 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.552306 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.553546 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.553554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.553584 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:24.553750 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.553757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555234 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.555237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.556670 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.556682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557764 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.557768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.557857 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.557879 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.557937 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.558113 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":138,"thread_start_us":79,"threads_started":1}
I20260811 05:12:24.558334 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.551971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:37053 (local address 127.0.0.1:40692)
0811 05:12:24.552761 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.552775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.552865 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.553750 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.553753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.553765 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:24.553872 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.553878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555148 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.555153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.556551 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.556566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557947 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.557952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.558043 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.558066 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.558134 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.558199 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":62,"thread_start_us":106,"threads_started":1}
I20260811 05:12:24.558450 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.552856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.30.88.193:44741)
0811 05:12:24.553255 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.553270 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.553719 (+ 449us) server_negotiation.cc:407] Connection header received
0811 05:12:24.553794 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.553799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.554003 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.554157 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:24.554412 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555451 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.556977 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557527 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.557656 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.557871 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.558199 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.558304 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":104,"threads_started":1}
I20260811 05:12:24.558732 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.552705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:44741 (local address 127.0.0.1:36870)
0811 05:12:24.553545 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.553558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.553644 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.554175 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.554178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.554200 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:24.554289 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.554295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555628 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.555631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.556804 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.556812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557669 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.557672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.557781 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.557820 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.557943 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.558105 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260811 05:12:24.558514 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.551903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.30.88.194:43561)
0811 05:12:24.553111 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.553124 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.553144 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:12:24.553208 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.553212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.553386 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.553556 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:24.553875 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555123 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.556811 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557610 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.558070 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.558202 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.558267 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.558365 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1110,"thread_start_us":132,"threads_started":1}
I20260811 05:12:24.558980 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.552547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.30.88.195:37053)
0811 05:12:24.553056 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.553073 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.553097 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:12:24.553179 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.553184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.553386 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.553549 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:24.553994 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.555006 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.556666 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.557610 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.557763 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.557917 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.558696 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.558799 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":171,"threads_started":1}
I20260811 05:12:24.561517 18754 tablet_service.cc:1467] Processing CreateTablet for tablet d30a95947d1345248611d73d5623190c (DEFAULT_TABLE table=test-table [id=c532005d288d4bab83b2ee8aec8c6832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:24.561514 18598 tablet_service.cc:1467] Processing CreateTablet for tablet d30a95947d1345248611d73d5623190c (DEFAULT_TABLE table=test-table [id=c532005d288d4bab83b2ee8aec8c6832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:24.562475 18754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30a95947d1345248611d73d5623190c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:24.562520 18598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30a95947d1345248611d73d5623190c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:24.563841 18436 tablet_service.cc:1467] Processing CreateTablet for tablet d30a95947d1345248611d73d5623190c (DEFAULT_TABLE table=test-table [id=c532005d288d4bab83b2ee8aec8c6832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:24.564771 18436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30a95947d1345248611d73d5623190c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:24.568579 18847 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Bootstrap starting.
I20260811 05:12:24.569962 18848 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Bootstrap starting.
I20260811 05:12:24.571003 18847 tablet_bootstrap.cc:654] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:24.571979 18849 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: Bootstrap starting.
I20260811 05:12:24.572191 18848 tablet_bootstrap.cc:654] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:24.572315 18847 log.cc:824] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:24.573309 18848 log.cc:824] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:24.574280 18849 tablet_bootstrap.cc:654] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:24.574457 18847 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: No bootstrap required, opened a new log
I20260811 05:12:24.574644 18847 ts_tablet_manager.cc:1397] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 05:12:24.575301 18848 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: No bootstrap required, opened a new log
I20260811 05:12:24.575474 18848 ts_tablet_manager.cc:1397] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:24.575449 18849 log.cc:824] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:24.577394 18849 tablet_bootstrap.cc:492] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: No bootstrap required, opened a new log
I20260811 05:12:24.577589 18849 ts_tablet_manager.cc:1397] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:24.578681 18847 raft_consensus.cc:348] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.578970 18847 raft_consensus.cc:374] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:24.579030 18847 raft_consensus.cc:729] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc75ea34b4cf498da2911e7cc1ca5ac6, State: Initialized, Role: FOLLOWER
I20260811 05:12:24.579298 18848 raft_consensus.cc:348] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.579530 18848 raft_consensus.cc:374] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:24.579479 18847 consensus_queue.cc:260] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.579598 18848 raft_consensus.cc:729] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bedd0f9ddd694ac7b7a3aa9c28e2fa83, State: Initialized, Role: FOLLOWER
I20260811 05:12:24.580157 18828 heartbeater.cc:503] Master 127.30.88.254:34951 was elected leader, sending a full tablet report...
I20260811 05:12:24.580467 18847 ts_tablet_manager.cc:1428] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:12:24.580883 18848 consensus_queue.cc:260] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.581511 18672 heartbeater.cc:503] Master 127.30.88.254:34951 was elected leader, sending a full tablet report...
I20260811 05:12:24.581472 18849 raft_consensus.cc:348] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.581708 18849 raft_consensus.cc:374] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:24.581764 18849 raft_consensus.cc:729] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25d702d825e4aacb7b4c7a6f7119eef, State: Initialized, Role: FOLLOWER
I20260811 05:12:24.581784 18848 ts_tablet_manager.cc:1428] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:24.582198 18849 consensus_queue.cc:260] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.582834 18516 heartbeater.cc:503] Master 127.30.88.254:34951 was elected leader, sending a full tablet report...
I20260811 05:12:24.583130 18849 ts_tablet_manager.cc:1428] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260811 05:12:24.632759 18854 raft_consensus.cc:482] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:24.632954 18854 raft_consensus.cc:504] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.634146 18854 leader_election.cc:288] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bedd0f9ddd694ac7b7a3aa9c28e2fa83 (127.30.88.194:43561), f25d702d825e4aacb7b4c7a6f7119eef (127.30.88.193:44741)
I20260811 05:12:24.637737 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.634370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:43561 (local address 127.30.88.195:49583)
0811 05:12:24.634548 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.634575 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.634686 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.634972 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.634976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.634998 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:24.635127 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.635134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.635857 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.635861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.636650 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.636659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.637304 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.637308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.637424 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.637442 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.637531 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.637606 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 05:12:24.637737 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.634524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:49583 (local address 127.30.88.194:43561)
0811 05:12:24.634692 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.634697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.634714 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:12:24.634784 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.634790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.634866 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.634943 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:24.635241 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.635741 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.636779 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.637181 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.637309 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.637361 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.637554 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.637620 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 05:12:24.638458 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.634651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:44741 (local address 127.30.88.195:57691)
0811 05:12:24.635017 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.635040 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:24.635167 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:24.635472 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:24.635475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:24.635493 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:24.635634 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.635640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.636389 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.636394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.637195 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:24.637205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.638065 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.638069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:24.638184 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.638217 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:12:24.638295 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:12:24.638352 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":133,"threads_started":1}
I20260811 05:12:24.638413 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30a95947d1345248611d73d5623190c" candidate_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" is_pre_election: true
I20260811 05:12:24.638535 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.634796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:57691 (local address 127.30.88.193:44741)
0811 05:12:24.634967 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.634973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.635128 (+ 155us) server_negotiation.cc:407] Connection header received
0811 05:12:24.635290 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.635293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.635352 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.635441 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:24.635760 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.636260 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.637336 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.637749 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.637921 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.637983 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.638338 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.638416 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 05:12:24.638724 18618 raft_consensus.cc:2454] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc75ea34b4cf498da2911e7cc1ca5ac6 in term 0.
I20260811 05:12:24.639160 18461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30a95947d1345248611d73d5623190c" candidate_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" is_pre_election: true
I20260811 05:12:24.639264 18701 leader_election.cc:302] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bedd0f9ddd694ac7b7a3aa9c28e2fa83, cc75ea34b4cf498da2911e7cc1ca5ac6; no voters:
I20260811 05:12:24.639461 18461 raft_consensus.cc:2454] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc75ea34b4cf498da2911e7cc1ca5ac6 in term 0.
I20260811 05:12:24.639593 18854 raft_consensus.cc:2781] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:24.639704 18854 raft_consensus.cc:482] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:24.639765 18854 raft_consensus.cc:3037] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:24.640796 18854 raft_consensus.cc:504] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.641127 18854 leader_election.cc:288] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [CANDIDATE]: Term 1 election: Requested vote from peers bedd0f9ddd694ac7b7a3aa9c28e2fa83 (127.30.88.194:43561), f25d702d825e4aacb7b4c7a6f7119eef (127.30.88.193:44741)
I20260811 05:12:24.641640 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30a95947d1345248611d73d5623190c" candidate_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83"
I20260811 05:12:24.641640 18461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30a95947d1345248611d73d5623190c" candidate_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25d702d825e4aacb7b4c7a6f7119eef"
I20260811 05:12:24.641754 18461 raft_consensus.cc:3037] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:24.641754 18618 raft_consensus.cc:3037] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:24.643042 18618 raft_consensus.cc:2454] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc75ea34b4cf498da2911e7cc1ca5ac6 in term 1.
I20260811 05:12:24.643050 18461 raft_consensus.cc:2454] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc75ea34b4cf498da2911e7cc1ca5ac6 in term 1.
I20260811 05:12:24.643381 18701 leader_election.cc:302] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc75ea34b4cf498da2911e7cc1ca5ac6, f25d702d825e4aacb7b4c7a6f7119eef; no voters:
I20260811 05:12:24.643589 18854 raft_consensus.cc:2781] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:24.643878 18854 raft_consensus.cc:686] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 1 LEADER]: Becoming Leader. State: Replica: cc75ea34b4cf498da2911e7cc1ca5ac6, State: Running, Role: LEADER
I20260811 05:12:24.644222 18854 consensus_queue.cc:237] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } }
I20260811 05:12:24.647523 18292 catalog_manager.cc:5495] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc75ea34b4cf498da2911e7cc1ca5ac6 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc75ea34b4cf498da2911e7cc1ca5ac6" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:24.658667 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.655007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.30.88.195:37053)
0811 05:12:24.655126 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.655130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.655215 (+ 85us) server_negotiation.cc:407] Connection header received
0811 05:12:24.655396 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.655403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.655454 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.655562 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:24.655807 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.656296 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.657245 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.657448 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.657555 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.657858 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:24.658276 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:24.658404 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.658464 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.658516 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:12:24.663887 18461 raft_consensus.cc:1261] T d30a95947d1345248611d73d5623190c P f25d702d825e4aacb7b4c7a6f7119eef [term 1 FOLLOWER]: Refusing update from remote peer cc75ea34b4cf498da2911e7cc1ca5ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:24.663897 18618 raft_consensus.cc:1261] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 1 FOLLOWER]: Refusing update from remote peer cc75ea34b4cf498da2911e7cc1ca5ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:24.664568 18854 consensus_queue.cc:1035] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bedd0f9ddd694ac7b7a3aa9c28e2fa83" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 43561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:24.664772 18858 consensus_queue.cc:1035] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25d702d825e4aacb7b4c7a6f7119eef" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:24.680918 18292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:24.681093 18292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:24.683472 18292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e340a58ea88746d4bf5a6a913df0979b: Sending DeleteTablet for 3 replicas of tablet d30a95947d1345248611d73d5623190c
I20260811 05:12:24.684230 18754 tablet_service.cc:1514] Processing DeleteTablet for tablet d30a95947d1345248611d73d5623190c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:40692
I20260811 05:12:24.684331 18436 tablet_service.cc:1514] Processing DeleteTablet for tablet d30a95947d1345248611d73d5623190c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:36870
I20260811 05:12:24.684734 18873 tablet_replica.cc:330] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: stopping tablet replica
I20260811 05:12:24.684728 18598 tablet_service.cc:1514] Processing DeleteTablet for tablet d30a95947d1345248611d73d5623190c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:24 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:34660
I20260811 05:12:24.685047 18873 raft_consensus.cc:2229] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:24.685202 18876 tablet_replica.cc:330] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: stopping tablet replica
I20260811 05:12:24.685367 18873 raft_consensus.cc:2258] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:24.685544 18876 raft_consensus.cc:2229] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:24.685784 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18367
I20260811 05:12:24.685894 18876 raft_consensus.cc:2258] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:24.688190 18876 ts_tablet_manager.cc:1905] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:24.688217 18873 ts_tablet_manager.cc:1905] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:24.690876 18876 ts_tablet_manager.cc:1918] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:24.691020 18876 log.cc:1196] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/d30a95947d1345248611d73d5623190c
I20260811 05:12:24.691404 18876 ts_tablet_manager.cc:1939] T d30a95947d1345248611d73d5623190c P bedd0f9ddd694ac7b7a3aa9c28e2fa83: Deleting consensus metadata
I20260811 05:12:24.691533 18873 ts_tablet_manager.cc:1918] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:24.691649 18873 log.cc:1196] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/d30a95947d1345248611d73d5623190c
I20260811 05:12:24.691954 18873 ts_tablet_manager.cc:1939] T d30a95947d1345248611d73d5623190c P cc75ea34b4cf498da2911e7cc1ca5ac6: Deleting consensus metadata
I20260811 05:12:24.692559 18282 catalog_manager.cc:4841] TS bedd0f9ddd694ac7b7a3aa9c28e2fa83 (127.30.88.194:43561): tablet d30a95947d1345248611d73d5623190c (table test-table [id=c532005d288d4bab83b2ee8aec8c6832]) successfully deleted
I20260811 05:12:24.692961 18279 catalog_manager.cc:4841] TS cc75ea34b4cf498da2911e7cc1ca5ac6 (127.30.88.195:37053): tablet d30a95947d1345248611d73d5623190c (table test-table [id=c532005d288d4bab83b2ee8aec8c6832]) successfully deleted
W20260811 05:12:24.698254 18282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:44741 (error 108)
I20260811 05:12:24.698431 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18521
I20260811 05:12:24.702301 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.702015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:44741 (local address 127.0.0.1:36874)
0811 05:12:24.702151 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:24.702221 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:44741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 05:12:24.702457 18282 catalog_manager.cc:4584] TS f25d702d825e4aacb7b4c7a6f7119eef (127.30.88.193:44741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d30a95947d1345248611d73d5623190c: Network error: Client connection negotiation failed: client connection to 127.30.88.193:44741: connect: Connection refused (error 111)
I20260811 05:12:24.706418 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18679
I20260811 05:12:24.712780 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18260
2026-08-11T05:12:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2384 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T05:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:24Z Disabled control of system clock
I20260811 05:12:24.746692 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:40813
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:40813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:24.856724 18882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:24.856964 18882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:24.856997 18882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:24.860806 18882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:24.860853 18882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:24.860868 18882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:24.860884 18882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:24.860900 18882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:24.865337 18882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:40813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:40813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:24.866161 18882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:24.867467 18882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:24.875345 18888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.875415 18889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:24.875406 18891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:24.875712 18882 server_base.cc:1034] running on GCE node
I20260811 05:12:24.876221 18882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:24.877206 18882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:24.878383 18882 hybrid_clock.cc:648] HybridClock initialized: now 1786425144878357 us; error 37 us; skew 500 ppm
I20260811 05:12:24.880208 18882 webserver.cc:460] Webserver started at http://127.30.88.254:35181/ using document root <none> and password file <none>
I20260811 05:12:24.880755 18882 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:24.880815 18882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:24.881131 18882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:24.883214 18882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "a4adda54d0c54243ba5469f5293df3cc"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "af4fd01b5167a05318237890252bab71"
server_key_iv: "5a568891bd2775d39c03f6968f28a0f1"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.883745 18882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "a4adda54d0c54243ba5469f5293df3cc"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "af4fd01b5167a05318237890252bab71"
server_key_iv: "5a568891bd2775d39c03f6968f28a0f1"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.887060 18882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:12:24.889230 18897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:24.890326 18882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:24.890466 18882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "a4adda54d0c54243ba5469f5293df3cc"
format_stamp: "Formatted at 2026-08-11 05:12:24 on dist-test-slave-7c35"
server_key: "af4fd01b5167a05318237890252bab71"
server_key_iv: "5a568891bd2775d39c03f6968f28a0f1"
server_key_version: "encryptionkey@0"
I20260811 05:12:24.890568 18882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:24.921425 18882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:24.922161 18882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:24.922362 18882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:24.929132 18882 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:40813
I20260811 05:12:24.929142 18957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:40813 every 8 connection(s)
I20260811 05:12:24.929996 18882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:24.932492 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18882
I20260811 05:12:24.932590 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:24.932895 31075 external_mini_cluster.cc:1428] Setting key 8565fa317b4d8a79320952ba0f01815b
I20260811 05:12:24.933185 18958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:24.939407 18958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: Bootstrap starting.
I20260811 05:12:24.941672 18958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:24.942041 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:24.934240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.30.88.254:40813)
0811 05:12:24.934947 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:24.934962 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:24.934997 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:24.935889 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:24.935906 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:24.936258 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:24.936586 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:24.937006 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.938003 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.938891 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:24.939348 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:24.939513 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:24.940407 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:24.940412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:24.940423 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:24.940781 (+ 358us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:24.940831 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:24.940849 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:24.940954 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:24.941087 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:12:24.941315 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260811 05:12:24.942441 18958 log.cc:824] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:24.944279 18958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: No bootstrap required, opened a new log
I20260811 05:12:24.947656 18958 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } }
I20260811 05:12:24.947880 18958 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:24.947959 18958 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4adda54d0c54243ba5469f5293df3cc, State: Initialized, Role: FOLLOWER
I20260811 05:12:24.948427 18958 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } }
I20260811 05:12:24.948627 18958 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:24.948700 18958 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:24.948802 18958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:24.949594 18958 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } }
I20260811 05:12:24.949910 18958 leader_election.cc:302] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4adda54d0c54243ba5469f5293df3cc; no voters:
I20260811 05:12:24.950210 18958 leader_election.cc:288] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:24.950354 18965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:24.950608 18965 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [term 1 LEADER]: Becoming Leader. State: Replica: a4adda54d0c54243ba5469f5293df3cc, State: Running, Role: LEADER
I20260811 05:12:24.950934 18965 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } }
I20260811 05:12:24.951467 18958 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:24.952492 18966 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4adda54d0c54243ba5469f5293df3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } } }
I20260811 05:12:24.952620 18966 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:24.952914 18967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4adda54d0c54243ba5469f5293df3cc. Latest consensus state: current_term: 1 leader_uuid: "a4adda54d0c54243ba5469f5293df3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4adda54d0c54243ba5469f5293df3cc" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 40813 } } }
I20260811 05:12:24.953014 18975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:24.953016 18967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:24.955716 18975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:24.960677 18975 catalog_manager.cc:1347] Generated new cluster ID: acc09a894eb04fd78119c4a60f1cd6d2
I20260811 05:12:24.960734 18975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:24.974970 18975 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:24.976117 18975 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:24.984310 18975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: Generated new TSK 0
I20260811 05:12:24.984978 18975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:24.992012 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--builtin_ntp_servers=127.30.88.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:25.102820 18987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:25.103084 18987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:25.103147 18987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:25.106778 18987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:25.106844 18987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:25.106946 18987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:25.111579 18987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:25.112495 18987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:25.113672 18987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:25.122550 18997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:25.122566 18995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:25.122550 18994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:25.122948 18987 server_base.cc:1034] running on GCE node
I20260811 05:12:25.123360 18987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:25.123934 18987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:25.125123 18987 hybrid_clock.cc:648] HybridClock initialized: now 1786425145125098 us; error 37 us; skew 500 ppm
I20260811 05:12:25.126832 18987 webserver.cc:460] Webserver started at http://127.30.88.193:36407/ using document root <none> and password file <none>
I20260811 05:12:25.127432 18987 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:25.127528 18987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:25.127740 18987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:25.129626 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "1243ada49ac241c5a05d3700421e28dc"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "0dca8f61da12b51e8b4d51fd8083a0b8"
server_key_iv: "be135e7df57939c44163e95cab1e844f"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.130157 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "1243ada49ac241c5a05d3700421e28dc"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "0dca8f61da12b51e8b4d51fd8083a0b8"
server_key_iv: "be135e7df57939c44163e95cab1e844f"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.133771 18987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 05:12:25.136142 19003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.137236 18987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:25.137409 18987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "1243ada49ac241c5a05d3700421e28dc"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "0dca8f61da12b51e8b4d51fd8083a0b8"
server_key_iv: "be135e7df57939c44163e95cab1e844f"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.137555 18987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:25.152684 18987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:25.153301 18987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:25.153481 18987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:25.154071 18987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:25.155004 18987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:25.155076 18987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.155136 18987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:25.155189 18987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.163677 18987 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:34043
I20260811 05:12:25.163699 19133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:34043 every 8 connection(s)
I20260811 05:12:25.164481 18987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:25.168843 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 18987
I20260811 05:12:25.168993 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:25.169282 31075 external_mini_cluster.cc:1428] Setting key 27e0a54bf0389f34a1677bd7aaa98a92
I20260811 05:12:25.171211 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--builtin_ntp_servers=127.30.88.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:25.173879 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.166180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:38599 (local address 127.30.88.254:40813)
0811 05:12:25.166311 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.166322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.167324 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 05:12:25.168658 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.168662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.168726 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.168825 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:25.169777 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.170501 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.171823 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.172048 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.172154 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.173320 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:25.173322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:25.173324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:25.173336 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:25.173356 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:25.173363 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:25.173420 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.173668 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.173746 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:12:25.174949 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.166492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40813 (local address 127.30.88.193:38599)
0811 05:12:25.167175 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.167214 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.168451 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.169039 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.169050 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.169152 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:25.169625 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.169638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.170632 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.170635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.171703 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.171714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.172191 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.172194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.172358 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.173009 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:25.173012 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:25.173066 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:25.173162 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:25.173449 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:25.173460 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:25.173468 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.173617 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.173918 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260811 05:12:25.176108 19134 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40813
I20260811 05:12:25.176357 19134 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:25.176985 19134 heartbeater.cc:511] Master 127.30.88.254:40813 requested a full tablet report, sending...
I20260811 05:12:25.178622 18915 ts_manager.cc:194] Registered new tserver with Master: 1243ada49ac241c5a05d3700421e28dc (127.30.88.193:34043)
I20260811 05:12:25.179811 18915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.193:38599
W20260811 05:12:25.280148 19137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:25.280359 19137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:25.280393 19137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:25.284381 19137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:25.284425 19137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:25.284512 19137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:25.289239 19137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:25.290134 19137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:25.291435 19137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:25.300655 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:25.300712 19143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:25.300647 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:25.300855 19137 server_base.cc:1034] running on GCE node
I20260811 05:12:25.301244 19137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:25.301766 19137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:25.302923 19137 hybrid_clock.cc:648] HybridClock initialized: now 1786425145302902 us; error 54 us; skew 500 ppm
I20260811 05:12:25.304651 19137 webserver.cc:460] Webserver started at http://127.30.88.194:34497/ using document root <none> and password file <none>
I20260811 05:12:25.305194 19137 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:25.305297 19137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:25.305512 19137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:25.307440 19137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "28254421ebc84160970cb9a26911f355"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "1783cb2007d58848e351407358d2b5ee"
server_key_iv: "c3f88bdc7001cf253bf9055db08e684d"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.308053 19137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "28254421ebc84160970cb9a26911f355"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "1783cb2007d58848e351407358d2b5ee"
server_key_iv: "c3f88bdc7001cf253bf9055db08e684d"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.311605 19137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:25.313786 19153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.314839 19137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:25.315001 19137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "28254421ebc84160970cb9a26911f355"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "1783cb2007d58848e351407358d2b5ee"
server_key_iv: "c3f88bdc7001cf253bf9055db08e684d"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.315101 19137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:25.331704 19137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:25.332293 19137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:25.332507 19137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:25.333050 19137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:25.333936 19137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:25.334005 19137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.334069 19137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:25.334117 19137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.342594 19137 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:37209
I20260811 05:12:25.342660 19288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:37209 every 8 connection(s)
I20260811 05:12:25.343474 19137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:25.348774 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19137
I20260811 05:12:25.348905 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:25.349192 31075 external_mini_cluster.cc:1428] Setting key 3da9e10a2dffa262c97b6a5972f89fc4
I20260811 05:12:25.351244 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--builtin_ntp_servers=127.30.88.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:25.353490 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.345256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:37989 (local address 127.30.88.254:40813)
0811 05:12:25.345426 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.345439 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.346369 (+ 930us) server_negotiation.cc:407] Connection header received
0811 05:12:25.347807 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.347810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.347858 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.347931 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:25.348817 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.349569 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.350902 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.351264 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.351390 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.352920 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:25.352923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:25.352925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:25.352937 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:25.352957 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:25.352965 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:25.353024 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.353219 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.353293 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 05:12:25.354439 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.345548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40813 (local address 127.30.88.194:37989)
0811 05:12:25.346231 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.346270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.347614 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.348086 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.348092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.348169 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:25.348673 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.348685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.349708 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.349714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.350761 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.350771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.351606 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.351612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.351847 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.352629 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:25.352633 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:25.352685 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:25.352763 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:25.353056 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:25.353067 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:25.353075 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.353207 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.353515 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":114,"threads_started":1}
I20260811 05:12:25.355523 19289 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40813
I20260811 05:12:25.355762 19289 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:25.356325 19289 heartbeater.cc:511] Master 127.30.88.254:40813 requested a full tablet report, sending...
I20260811 05:12:25.357450 18915 ts_manager.cc:194] Registered new tserver with Master: 28254421ebc84160970cb9a26911f355 (127.30.88.194:37209)
I20260811 05:12:25.358006 18915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.194:37989
W20260811 05:12:25.462100 19292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:25.462324 19292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:25.462358 19292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:25.466274 19292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:25.466320 19292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:25.466365 19292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:25.471235 19292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:25.472177 19292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:25.473479 19292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:25.482668 19297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:25.482717 19292 server_base.cc:1034] running on GCE node
W20260811 05:12:25.482668 19298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:25.482677 19300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:25.483263 19292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:25.483807 19292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:25.484956 19292 hybrid_clock.cc:648] HybridClock initialized: now 1786425145484927 us; error 38 us; skew 500 ppm
I20260811 05:12:25.486588 19292 webserver.cc:460] Webserver started at http://127.30.88.195:37939/ using document root <none> and password file <none>
I20260811 05:12:25.487160 19292 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:25.487241 19292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:25.487457 19292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:25.489339 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "5875789064f34b5aac68be398ecf4561"
server_key_iv: "98aaeeef9b3efe5e32b2fb1b687e913c"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.489872 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "5875789064f34b5aac68be398ecf4561"
server_key_iv: "98aaeeef9b3efe5e32b2fb1b687e913c"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.493741 19292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:12:25.496070 19306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.497347 19292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:25.497479 19292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f"
format_stamp: "Formatted at 2026-08-11 05:12:25 on dist-test-slave-7c35"
server_key: "5875789064f34b5aac68be398ecf4561"
server_key_iv: "98aaeeef9b3efe5e32b2fb1b687e913c"
server_key_version: "encryptionkey@0"
I20260811 05:12:25.497578 19292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:25.519519 19292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:25.520152 19292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:25.520340 19292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:25.520905 19292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:25.521826 19292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:25.521871 19292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.521939 19292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:25.521974 19292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:25.530638 19292 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:40927
I20260811 05:12:25.530674 19439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:40927 every 8 connection(s)
I20260811 05:12:25.531479 19292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:25.539489 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19292
I20260811 05:12:25.539644 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:25.539934 31075 external_mini_cluster.cc:1428] Setting key 725f52ba4ed9617086429413a4e56f4b
I20260811 05:12:25.540699 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.533136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:48129 (local address 127.30.88.254:40813)
0811 05:12:25.533298 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.533301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.534214 (+ 913us) server_negotiation.cc:407] Connection header received
0811 05:12:25.535676 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.535680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.535740 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.535847 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:25.536697 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.537239 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.538425 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.538621 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.538727 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.539965 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:25.539968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:25.539970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:25.540001 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:25.540021 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:25.540029 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:25.540113 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.540500 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.540558 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 05:12:25.541558 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.533471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:40813 (local address 127.30.88.195:48129)
0811 05:12:25.534074 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.534111 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.535464 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.536022 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.536028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.536118 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:25.536555 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.536566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.537381 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.537384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.538243 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.538253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.538764 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.538768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.538977 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.539592 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:25.539593 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:25.539671 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:25.539771 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:25.540129 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:25.540138 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:25.540144 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.540293 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.540566 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260811 05:12:25.542616 19440 heartbeater.cc:347] Connected to a master server at 127.30.88.254:40813
I20260811 05:12:25.542832 19440 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:25.543346 19440 heartbeater.cc:511] Master 127.30.88.254:40813 requested a full tablet report, sending...
I20260811 05:12:25.544246 18914 ts_manager.cc:194] Registered new tserver with Master: 5ceefedbf336489fb3ba1b3d7fce1c7f (127.30.88.195:40927)
I20260811 05:12:25.544775 18914 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.195:48129
I20260811 05:12:25.553972 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:25.559466 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.555966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.30.88.254:40813)
0811 05:12:25.556105 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.556108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.556292 (+ 184us) server_negotiation.cc:407] Connection header received
0811 05:12:25.556382 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.556385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.556424 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.556491 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:25.556742 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.557235 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.558065 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.558264 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.558337 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.559106 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:25.559109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:25.559111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:25.559135 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:25.559152 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:25.559160 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:25.559216 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.559322 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.559369 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:12:25.562247 18914 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:25.565428 18914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:25.580508 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.575173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:40927 (local address 127.0.0.1:35802)
0811 05:12:25.575677 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.575690 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.575773 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.576512 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.576515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.576529 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:25.576640 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.576646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.578120 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.578123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.579154 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.579164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.579958 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.579961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.580092 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.580133 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.580233 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.580349 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":16,"thread_start_us":116,"threads_started":1}
I20260811 05:12:25.580575 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.575352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.30.88.195:40927)
0811 05:12:25.576018 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.576030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.576052 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:12:25.576124 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.576133 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.576307 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.576516 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:25.576756 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.577954 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.579281 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.579805 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.580059 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.580255 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.580305 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.580409 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":153,"threads_started":1}
I20260811 05:12:25.580705 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.574384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:37209 (local address 127.0.0.1:43748)
0811 05:12:25.575183 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.575246 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.575431 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.576268 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.576274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.576296 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:25.576492 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.576501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.578082 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.578086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.579223 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.579233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.580251 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.580255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.580348 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.580370 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.580446 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.580518 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260811 05:12:25.580818 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.574521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43748 (local address 127.30.88.194:37209)
0811 05:12:25.575714 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.575730 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.575756 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:12:25.575840 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.575860 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.576073 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.576634 (+ 561us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:25.576799 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.577959 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.579355 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.580110 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.580238 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.580416 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.580471 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.580577 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1092,"thread_start_us":135,"threads_started":1}
I20260811 05:12:25.581398 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.574421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34043 (local address 127.0.0.1:48692)
0811 05:12:25.575529 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.575543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.575646 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.576357 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.576360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.576393 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:25.576522 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.576529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.577597 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.577602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.578820 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.578831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.579920 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.579926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.581166 (+ 1240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.581187 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.581260 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.581305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":28,"thread_start_us":100,"threads_started":1}
I20260811 05:12:25.581698 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.575044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48692 (local address 127.30.88.193:34043)
0811 05:12:25.575511 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.575524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.575756 (+ 232us) server_negotiation.cc:407] Connection header received
0811 05:12:25.575844 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.575850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.576075 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.576236 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:25.576657 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.577476 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.578988 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.579777 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.579890 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.580091 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.581317 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.581415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":130,"threads_started":1}
I20260811 05:12:25.584105 19211 tablet_service.cc:1467] Processing CreateTablet for tablet c609305273d345c38d9de851a7946e7b (DEFAULT_TABLE table=test-table [id=aff71a4931ca4b21bac64ee31fe1589d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:25.584251 19058 tablet_service.cc:1467] Processing CreateTablet for tablet c609305273d345c38d9de851a7946e7b (DEFAULT_TABLE table=test-table [id=aff71a4931ca4b21bac64ee31fe1589d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:25.584609 19364 tablet_service.cc:1467] Processing CreateTablet for tablet c609305273d345c38d9de851a7946e7b (DEFAULT_TABLE table=test-table [id=aff71a4931ca4b21bac64ee31fe1589d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:25.585170 19211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c609305273d345c38d9de851a7946e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:25.585259 19058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c609305273d345c38d9de851a7946e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:25.585472 19364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c609305273d345c38d9de851a7946e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:25.591524 19459 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Bootstrap starting.
I20260811 05:12:25.593068 19461 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Bootstrap starting.
I20260811 05:12:25.593509 19460 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Bootstrap starting.
I20260811 05:12:25.594873 19459 tablet_bootstrap.cc:654] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:25.595109 19461 tablet_bootstrap.cc:654] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:25.595656 19460 tablet_bootstrap.cc:654] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:25.595788 19459 log.cc:824] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:25.595800 19461 log.cc:824] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:25.596493 19460 log.cc:824] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:25.597908 19461 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: No bootstrap required, opened a new log
I20260811 05:12:25.597910 19459 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: No bootstrap required, opened a new log
I20260811 05:12:25.598102 19461 ts_tablet_manager.cc:1397] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:12:25.598097 19459 ts_tablet_manager.cc:1397] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 05:12:25.598508 19460 tablet_bootstrap.cc:492] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: No bootstrap required, opened a new log
I20260811 05:12:25.598701 19460 ts_tablet_manager.cc:1397] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:12:25.601732 19461 raft_consensus.cc:348] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.601989 19461 raft_consensus.cc:374] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:25.602134 19461 raft_consensus.cc:729] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1243ada49ac241c5a05d3700421e28dc, State: Initialized, Role: FOLLOWER
I20260811 05:12:25.602352 19459 raft_consensus.cc:348] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.602624 19459 raft_consensus.cc:374] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:25.602576 19461 consensus_queue.cc:260] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.602692 19459 raft_consensus.cc:729] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28254421ebc84160970cb9a26911f355, State: Initialized, Role: FOLLOWER
I20260811 05:12:25.602895 19460 raft_consensus.cc:348] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.603137 19460 raft_consensus.cc:374] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:25.603188 19460 raft_consensus.cc:729] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ceefedbf336489fb3ba1b3d7fce1c7f, State: Initialized, Role: FOLLOWER
I20260811 05:12:25.603135 19459 consensus_queue.cc:260] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.603683 19461 ts_tablet_manager.cc:1428] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:12:25.603615 19460 consensus_queue.cc:260] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.604063 19459 ts_tablet_manager.cc:1428] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:12:25.604072 19134 heartbeater.cc:503] Master 127.30.88.254:40813 was elected leader, sending a full tablet report...
I20260811 05:12:25.604401 19289 heartbeater.cc:503] Master 127.30.88.254:40813 was elected leader, sending a full tablet report...
I20260811 05:12:25.604501 19460 ts_tablet_manager.cc:1428] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:12:25.604840 19440 heartbeater.cc:503] Master 127.30.88.254:40813 was elected leader, sending a full tablet report...
W20260811 05:12:25.665604 19135 tablet.cc:2369] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:25.782440 19441 tablet.cc:2369] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:25.844560 19290 tablet.cc:2369] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:25.932552 19466 raft_consensus.cc:482] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:25.932777 19466 raft_consensus.cc:504] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.934022 19466 leader_election.cc:288] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28254421ebc84160970cb9a26911f355 (127.30.88.194:37209), 5ceefedbf336489fb3ba1b3d7fce1c7f (127.30.88.195:40927)
I20260811 05:12:25.938298 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.934226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:37209 (local address 127.30.88.193:33745)
0811 05:12:25.934640 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.934655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.934825 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.935271 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.935273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.935301 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:25.935482 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.935489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.936347 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.936349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.937224 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.937234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.937887 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.937891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.937982 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.938007 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.938072 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.938136 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":9,"spinlock_wait_cycles":8832,"thread_start_us":105,"threads_started":1}
I20260811 05:12:25.938306 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.934342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:33745 (local address 127.30.88.194:37209)
0811 05:12:25.934488 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.934493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.934731 (+ 238us) server_negotiation.cc:407] Connection header received
0811 05:12:25.934993 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.934996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.935053 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.935105 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:25.935620 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.936231 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.937351 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.937751 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.937924 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.937978 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.938112 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.938182 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:12:25.939129 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.934636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:52775 (local address 127.30.88.195:40927)
0811 05:12:25.934757 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:25.934761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:25.934887 (+ 126us) server_negotiation.cc:407] Connection header received
0811 05:12:25.935071 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:25.935074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:25.935119 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:25.935163 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:25.935610 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.936341 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.937846 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.938502 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.938652 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.938698 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:25.938942 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 05:12:25.939018 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:12:25.939141 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:25.934288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:40927 (local address 127.30.88.193:52775)
0811 05:12:25.934766 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:25.934780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:25.934957 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:25.935279 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:25.935282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:25.935304 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:25.935489 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.935494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.936471 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.936473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.937696 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:25.937706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:25.938652 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:25.938655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:25.938759 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:25.938786 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:12:25.938922 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:12:25.939001 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":39,"spinlock_wait_cycles":1792,"thread_start_us":60,"threads_started":1}
I20260811 05:12:25.939249 19231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c609305273d345c38d9de851a7946e7b" candidate_uuid: "1243ada49ac241c5a05d3700421e28dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28254421ebc84160970cb9a26911f355" is_pre_election: true
I20260811 05:12:25.939599 19231 raft_consensus.cc:2454] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1243ada49ac241c5a05d3700421e28dc in term 0.
I20260811 05:12:25.939848 19386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c609305273d345c38d9de851a7946e7b" candidate_uuid: "1243ada49ac241c5a05d3700421e28dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" is_pre_election: true
I20260811 05:12:25.940106 19386 raft_consensus.cc:2454] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1243ada49ac241c5a05d3700421e28dc in term 0.
I20260811 05:12:25.940169 19005 leader_election.cc:302] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1243ada49ac241c5a05d3700421e28dc, 28254421ebc84160970cb9a26911f355; no voters:
I20260811 05:12:25.940512 19466 raft_consensus.cc:2781] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:25.940639 19466 raft_consensus.cc:482] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:25.940685 19466 raft_consensus.cc:3037] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:25.941926 19466 raft_consensus.cc:504] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.942278 19466 leader_election.cc:288] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [CANDIDATE]: Term 1 election: Requested vote from peers 28254421ebc84160970cb9a26911f355 (127.30.88.194:37209), 5ceefedbf336489fb3ba1b3d7fce1c7f (127.30.88.195:40927)
I20260811 05:12:25.942607 19386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c609305273d345c38d9de851a7946e7b" candidate_uuid: "1243ada49ac241c5a05d3700421e28dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f"
I20260811 05:12:25.942766 19386 raft_consensus.cc:3037] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:25.943457 19231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c609305273d345c38d9de851a7946e7b" candidate_uuid: "1243ada49ac241c5a05d3700421e28dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28254421ebc84160970cb9a26911f355"
I20260811 05:12:25.943626 19231 raft_consensus.cc:3037] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:25.943994 19386 raft_consensus.cc:2454] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1243ada49ac241c5a05d3700421e28dc in term 1.
I20260811 05:12:25.944324 19007 leader_election.cc:302] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1243ada49ac241c5a05d3700421e28dc, 5ceefedbf336489fb3ba1b3d7fce1c7f; no voters:
I20260811 05:12:25.944499 19466 raft_consensus.cc:2781] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:25.944751 19466 raft_consensus.cc:686] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 1 LEADER]: Becoming Leader. State: Replica: 1243ada49ac241c5a05d3700421e28dc, State: Running, Role: LEADER
I20260811 05:12:25.945012 19231 raft_consensus.cc:2454] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1243ada49ac241c5a05d3700421e28dc in term 1.
I20260811 05:12:25.945144 19466 consensus_queue.cc:237] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } }
I20260811 05:12:25.948376 18915 catalog_manager.cc:5495] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1243ada49ac241c5a05d3700421e28dc (127.30.88.193). New cstate: current_term: 1 leader_uuid: "1243ada49ac241c5a05d3700421e28dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1243ada49ac241c5a05d3700421e28dc" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 34043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:26.032292 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:26.028851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.30.88.193:34043)
0811 05:12:26.028989 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:26.028994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:26.029010 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:12:26.029153 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:26.029157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:26.029230 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:26.029335 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:26.029591 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:26.030112 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:26.030990 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:26.031207 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:26.031285 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:26.031573 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:26.031971 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:26.032070 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:26.032128 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 05:12:26.032172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:12:26.038116 19231 raft_consensus.cc:1261] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 1 FOLLOWER]: Refusing update from remote peer 1243ada49ac241c5a05d3700421e28dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:26.038231 19386 raft_consensus.cc:1261] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 1 FOLLOWER]: Refusing update from remote peer 1243ada49ac241c5a05d3700421e28dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:26.038908 19472 consensus_queue.cc:1035] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "28254421ebc84160970cb9a26911f355" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:26.039108 19466 consensus_queue.cc:1035] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ceefedbf336489fb3ba1b3d7fce1c7f" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 40927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:26.046502 19476 mvcc.cc:205] Tried to move back new op lower bound from 7317197398164066304 to 7317197397794082816. Current Snapshot: MvccSnapshot[applied={T|T < 7317197398164066304}]
I20260811 05:12:26.054955 18914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:26.055086 18914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:26.057340 18914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4adda54d0c54243ba5469f5293df3cc: Sending DeleteTablet for 3 replicas of tablet c609305273d345c38d9de851a7946e7b
I20260811 05:12:26.058040 19211 tablet_service.cc:1514] Processing DeleteTablet for tablet c609305273d345c38d9de851a7946e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:26 UTC) from {username='slave'} at 127.0.0.1:43748
I20260811 05:12:26.058053 19058 tablet_service.cc:1514] Processing DeleteTablet for tablet c609305273d345c38d9de851a7946e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:26 UTC) from {username='slave'} at 127.0.0.1:48692
I20260811 05:12:26.058166 19364 tablet_service.cc:1514] Processing DeleteTablet for tablet c609305273d345c38d9de851a7946e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:26 UTC) from {username='slave'} at 127.0.0.1:35802
I20260811 05:12:26.058521 19486 tablet_replica.cc:330] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc: stopping tablet replica
I20260811 05:12:26.058809 19488 tablet_replica.cc:330] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: stopping tablet replica
I20260811 05:12:26.058812 19486 raft_consensus.cc:2229] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:26.059104 19488 raft_consensus.cc:2229] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:26.059185 19486 raft_consensus.cc:2258] T c609305273d345c38d9de851a7946e7b P 1243ada49ac241c5a05d3700421e28dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:26.059185 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18987
I20260811 05:12:26.059347 19488 raft_consensus.cc:2258] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:26.060633 19488 ts_tablet_manager.cc:1905] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:26.062747 19488 ts_tablet_manager.cc:1918] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:26.062856 19488 log.cc:1196] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/c609305273d345c38d9de851a7946e7b
I20260811 05:12:26.063163 19488 ts_tablet_manager.cc:1939] T c609305273d345c38d9de851a7946e7b P 5ceefedbf336489fb3ba1b3d7fce1c7f: Deleting consensus metadata
I20260811 05:12:26.064173 19487 tablet_replica.cc:330] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: stopping tablet replica
I20260811 05:12:26.064162 18901 catalog_manager.cc:4841] TS 5ceefedbf336489fb3ba1b3d7fce1c7f (127.30.88.195:40927): tablet c609305273d345c38d9de851a7946e7b (table test-table [id=aff71a4931ca4b21bac64ee31fe1589d]) successfully deleted
I20260811 05:12:26.064419 19487 raft_consensus.cc:2229] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:26.064684 19487 raft_consensus.cc:2258] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:26.065816 19487 ts_tablet_manager.cc:1905] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:26.067824 19487 ts_tablet_manager.cc:1918] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:26.067904 19487 log.cc:1196] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/c609305273d345c38d9de851a7946e7b
I20260811 05:12:26.068176 19487 ts_tablet_manager.cc:1939] T c609305273d345c38d9de851a7946e7b P 28254421ebc84160970cb9a26911f355: Deleting consensus metadata
I20260811 05:12:26.069085 18899 catalog_manager.cc:4841] TS 28254421ebc84160970cb9a26911f355 (127.30.88.194:37209): tablet c609305273d345c38d9de851a7946e7b (table test-table [id=aff71a4931ca4b21bac64ee31fe1589d]) successfully deleted
W20260811 05:12:26.069439 18900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:34043 (error 108)
I20260811 05:12:26.070212 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19137
I20260811 05:12:26.070391 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:26.070095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:34043 (local address 127.0.0.1:48702)
0811 05:12:26.070217 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:26.070284 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:12:26.071261 18900 catalog_manager.cc:4584] TS 1243ada49ac241c5a05d3700421e28dc (127.30.88.193:34043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c609305273d345c38d9de851a7946e7b: Network error: Client connection negotiation failed: client connection to 127.30.88.193:34043: connect: Connection refused (error 111)
I20260811 05:12:26.079291 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19292
I20260811 05:12:26.088507 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 18882
2026-08-11T05:12:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1373 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:26 dist-test-slave-7c35 krb5kdc[19493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:26 dist-test-slave-7c35 krb5kdc[19493](info): set up 2 sockets
Aug 11 05:12:26 dist-test-slave-7c35 krb5kdc[19493](info): commencing operation
krb5kdc: starting...
W20260811 05:12:27.185639 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:27.309237 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35147
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:27.420274 19508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:27.420521 19508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:27.420567 19508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:27.424443 19508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:27.424496 19508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:27.424515 19508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:27.424531 19508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:27.424552 19508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:27.429647 19508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:27.430634 19508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:27.431651 19508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:27.440145 19517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:27.440243 19516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:27.440145 19519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:27.440369 19508 server_base.cc:1034] running on GCE node
I20260811 05:12:27.441118 19508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:27.442329 19508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:27.443531 19508 hybrid_clock.cc:648] HybridClock initialized: now 1786425147443517 us; error 39 us; skew 500 ppm
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:27.450793 19508 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:27.451742 19508 webserver.cc:460] Webserver started at http://127.30.88.254:37779/ using document root <none> and password file <none>
I20260811 05:12:27.452328 19508 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:27.452433 19508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:27.452670 19508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:27.454780 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9ec0b1d31fd97106ebfd101566f8d9b6"
server_key_iv: "17d0902bc7c336a6ba0b21c5674bc405"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.455435 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9ec0b1d31fd97106ebfd101566f8d9b6"
server_key_iv: "17d0902bc7c336a6ba0b21c5674bc405"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.459156 19508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:27.461498 19527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.462664 19508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:27.462807 19508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9ec0b1d31fd97106ebfd101566f8d9b6"
server_key_iv: "17d0902bc7c336a6ba0b21c5674bc405"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.462982 19508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:27.475879 19508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:27.479235 19508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:27.479447 19508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:27.486438 19508 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35147
I20260811 05:12:27.486515 19586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35147 every 8 connection(s)
I20260811 05:12:27.487498 19508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:27.490211 19587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:27.495610 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19508
I20260811 05:12:27.495726 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:27.495965 19587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: Bootstrap starting.
I20260811 05:12:27.496042 31075 external_mini_cluster.cc:1428] Setting key b4ea9bf935f35b2cc1d73a3f4cd2f39c
I20260811 05:12:27.498467 19587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:27.499230 19587 log.cc:824] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:27.501295 19587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: No bootstrap required, opened a new log
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:27.505745 19587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } }
I20260811 05:12:27.505983 19587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:27.506042 19587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea6dfbb9f634319b1ec33e0ffbcf416, State: Initialized, Role: FOLLOWER
I20260811 05:12:27.506505 19587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } }
I20260811 05:12:27.506664 19587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:27.506757 19587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:27.506896 19587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:27.508107 19587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } }
I20260811 05:12:27.508491 19587 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea6dfbb9f634319b1ec33e0ffbcf416; no voters:
I20260811 05:12:27.508849 19587 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:27.508976 19592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:27.509253 19592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea6dfbb9f634319b1ec33e0ffbcf416, State: Running, Role: LEADER
I20260811 05:12:27.509584 19592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } }
I20260811 05:12:27.509995 19587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:27.510918 19592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea6dfbb9f634319b1ec33e0ffbcf416. Latest consensus state: current_term: 1 leader_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } } }
I20260811 05:12:27.511044 19592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:27.511557 19602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:27.512009 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:27.497500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.30.88.254:35147)
0811 05:12:27.497950 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:27.497965 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:27.498000 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:27.498710 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:27.498731 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:27.499166 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:27.499503 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:27.500375 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.501407 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.502594 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.503140 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.503259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:27.506079 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:27.506087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:27.506105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:27.506143 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:27.508297 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:27.508766 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:27.508776 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:27.508784 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:27.508960 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:27.509343 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:27.509346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:27.509348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:27.509886 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:27.510058 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:27.510585 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 05:12:27.510947 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260811 05:12:27.514792 19602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:27.514752 19594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea6dfbb9f634319b1ec33e0ffbcf416" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35147 } } }
I20260811 05:12:27.514917 19594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:27.520555 19602 catalog_manager.cc:1347] Generated new cluster ID: 0d9e6984faf449a08a4dff9da928e2ea
I20260811 05:12:27.520640 19602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:27.528949 19602 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:27.529829 19602 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:27.538427 19602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: Generated new TSK 0
I20260811 05:12:27.539109 19602 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:12:27.570004 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:27.683902 19615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:27.684156 19615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:27.684196 19615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:27.688016 19615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:27.688066 19615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:27.688119 19615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:27.693367 19615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:27.694222 19615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:27.695129 19615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:27.704317 19622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:27.704325 19621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:27.704419 19615 server_base.cc:1034] running on GCE node
W20260811 05:12:27.704365 19624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:27.704916 19615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:27.705587 19615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:27.706779 19615 hybrid_clock.cc:648] HybridClock initialized: now 1786425147706752 us; error 28 us; skew 500 ppm
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:27.709501 19615 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:12:27.710464 19615 webserver.cc:460] Webserver started at http://127.30.88.193:38177/ using document root <none> and password file <none>
I20260811 05:12:27.711095 19615 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:27.711158 19615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:27.711356 19615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:27.713346 19615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "fd644ff1f11a4119815be5480767ba5c"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "2e64854d6b05083b46d1f0c0ed4e5020"
server_key_iv: "65c2098d722676044b4aba70e1f63cc8"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.713912 19615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "fd644ff1f11a4119815be5480767ba5c"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "2e64854d6b05083b46d1f0c0ed4e5020"
server_key_iv: "65c2098d722676044b4aba70e1f63cc8"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.717571 19615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 05:12:27.719970 19631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.721148 19615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:27.721251 19615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "fd644ff1f11a4119815be5480767ba5c"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "2e64854d6b05083b46d1f0c0ed4e5020"
server_key_iv: "65c2098d722676044b4aba70e1f63cc8"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.721374 19615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:27.765949 19615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:27.769137 19615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:27.769340 19615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:27.769936 19615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:27.770782 19615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:27.770829 19615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.770953 19615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:27.770988 19615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.779791 19615 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:36327
I20260811 05:12:27.779837 19762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:36327 every 8 connection(s)
I20260811 05:12:27.780654 19615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:27.786581 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19615
I20260811 05:12:27.786724 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:27.787056 31075 external_mini_cluster.cc:1428] Setting key 044eaf67412f22116cfbdaeac7647a0a
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:27.795336 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:27.782366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:54415 (local address 127.30.88.254:35147)
0811 05:12:27.782519 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:27.782523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:27.783353 (+ 830us) server_negotiation.cc:407] Connection header received
0811 05:12:27.784601 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:27.784604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:27.784650 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:27.784725 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:27.785781 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.786329 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.787940 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.788245 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.788365 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:27.791381 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:27.791385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:27.791387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:27.791408 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:27.793007 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:27.793525 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:27.793531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:27.793560 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:27.793652 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:27.793994 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:27.794001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:27.794005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:27.794259 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:27.794372 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:27.794931 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 05:12:27.795090 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:12:27.795933 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:27.782760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35147 (local address 127.30.88.193:54415)
0811 05:12:27.783210 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:27.783248 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:27.784409 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:27.784885 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:27.784891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:27.785215 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:27.785647 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:27.785660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.786457 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.786460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:27.787778 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:27.787789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.788418 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.788426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:27.788648 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:27.789221 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:27.789244 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:27.791186 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:27.793128 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:27.793133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:27.793144 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:27.793402 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:27.793739 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:27.793741 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:27.793743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:27.793869 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:27.794382 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:27.794403 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:27.794626 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 05:12:27.794773 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:12:27.795099 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":102,"threads_started":1}
I20260811 05:12:27.797133 19763 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35147
I20260811 05:12:27.797382 19763 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:27.797906 19763 heartbeater.cc:511] Master 127.30.88.254:35147 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:12:27.799515 19546 ts_manager.cc:194] Registered new tserver with Master: fd644ff1f11a4119815be5480767ba5c (127.30.88.193:36327)
I20260811 05:12:27.800652 19546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:54415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:12:27.817756 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:27.925176 19769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:27.925400 19769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:27.925474 19769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:27.929380 19769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:27.929427 19769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:27.929479 19769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:27.934800 19769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:27.935910 19769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:27.937191 19769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:27.946595 19780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:27.946591 19776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:27.946591 19777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:27.947224 19769 server_base.cc:1034] running on GCE node
I20260811 05:12:27.947635 19769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:27.948238 19769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:27.949400 19769 hybrid_clock.cc:648] HybridClock initialized: now 1786425147949386 us; error 25 us; skew 500 ppm
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:27.952347 19769 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:12:27.953258 19769 webserver.cc:460] Webserver started at http://127.30.88.194:32973/ using document root <none> and password file <none>
I20260811 05:12:27.953789 19769 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:27.953867 19769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:27.954118 19769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:27.956172 19769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "7eaa2443e771455d82d73c89ee0f9dab"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9c08791480c2abb31f5d692c395c657c"
server_key_iv: "ecabfd03df6bbb33e5387d51ba359754"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.956766 19769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "7eaa2443e771455d82d73c89ee0f9dab"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9c08791480c2abb31f5d692c395c657c"
server_key_iv: "ecabfd03df6bbb33e5387d51ba359754"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.960323 19769 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 05:12:27.962618 19788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.963891 19769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:27.964017 19769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "7eaa2443e771455d82d73c89ee0f9dab"
format_stamp: "Formatted at 2026-08-11 05:12:27 on dist-test-slave-7c35"
server_key: "9c08791480c2abb31f5d692c395c657c"
server_key_iv: "ecabfd03df6bbb33e5387d51ba359754"
server_key_version: "encryptionkey@0"
I20260811 05:12:27.964115 19769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:27.976198 19769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:27.979094 19769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:27.979291 19769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:27.979841 19769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:27.980620 19769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:27.980664 19769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.980731 19769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:27.980767 19769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:27.989535 19769 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:41971
I20260811 05:12:27.989573 19919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:41971 every 8 connection(s)
I20260811 05:12:27.990402 19769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:27.993932 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19769
I20260811 05:12:27.994036 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:27.994347 31075 external_mini_cluster.cc:1428] Setting key b622533eaae881993577430613764f56
Aug 11 05:12:27 dist-test-slave-7c35 krb5kdc[19493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425147, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:28.004880 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:27.992122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:33043 (local address 127.30.88.254:35147)
0811 05:12:27.992273 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:27.992276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:27.993005 (+ 729us) server_negotiation.cc:407] Connection header received
0811 05:12:27.994410 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:27.994413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:27.994491 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:27.994582 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:27.996068 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.996574 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.997787 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.998069 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.998154 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.001217 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:28.001224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:28.001231 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:28.001265 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:28.002722 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.003311 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.003314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.003316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.003412 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.003762 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.003765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.003767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.003961 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:28.004091 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.004578 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.004705 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
I20260811 05:12:28.005722 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:27.992437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35147 (local address 127.30.88.194:33043)
0811 05:12:27.992847 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:27.992896 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:27.994184 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:27.994750 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:27.994758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:27.995253 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:27.995896 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:27.995915 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.996713 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.996717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:27.997621 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:27.997630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:27.998226 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:27.998232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:27.998454 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:27.999232 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:27.999255 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:28.001014 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:28.002881 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:28.002887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:28.002927 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:28.003201 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:28.003510 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:28.003514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:28.003516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:28.003655 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:28.004077 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:28.004084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:28.004333 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.004516 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.004841 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260811 05:12:28.007097 19920 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35147
I20260811 05:12:28.007339 19920 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:28.007922 19920 heartbeater.cc:511] Master 127.30.88.254:35147 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
I20260811 05:12:28.009122 19546 ts_manager.cc:194] Registered new tserver with Master: 7eaa2443e771455d82d73c89ee0f9dab (127.30.88.194:41971)
I20260811 05:12:28.009711 19546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:33043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:12:28.023993 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:28.136853 19925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:28.137131 19925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:28.137178 19925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:28.141171 19925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:28.141232 19925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:28.141322 19925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:28.146786 19925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:28.147878 19925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:28.148907 19925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:28.158694 19933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:28.158680 19931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:28.158680 19930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:28.159070 19925 server_base.cc:1034] running on GCE node
I20260811 05:12:28.159489 19925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:28.160090 19925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:28.161271 19925 hybrid_clock.cc:648] HybridClock initialized: now 1786425148161258 us; error 39 us; skew 500 ppm
Aug 11 05:12:28 dist-test-slave-7c35 krb5kdc[19493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425148, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:28.164063 19925 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:12:28.164911 19925 webserver.cc:460] Webserver started at http://127.30.88.195:42895/ using document root <none> and password file <none>
I20260811 05:12:28.165444 19925 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:28.165530 19925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:28.165748 19925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:28.167904 19925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "2570b567654c4eb08edd4cf8bf07d398"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "c7426a566d360dc1f9f264a63d4a1b5c"
server_key_iv: "c12fd1ae56663dd107586b9a39ec2472"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.168504 19925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "2570b567654c4eb08edd4cf8bf07d398"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "c7426a566d360dc1f9f264a63d4a1b5c"
server_key_iv: "c12fd1ae56663dd107586b9a39ec2472"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.172106 19925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:12:28.174535 19941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.175737 19925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:28.175868 19925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "2570b567654c4eb08edd4cf8bf07d398"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "c7426a566d360dc1f9f264a63d4a1b5c"
server_key_iv: "c12fd1ae56663dd107586b9a39ec2472"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.175974 19925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:28.206130 19925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:28.209350 19925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:28.209550 19925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:28.210157 19925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:28.211098 19925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:28.211148 19925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.211189 19925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:28.211203 19925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.220180 19925 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:45295
I20260811 05:12:28.220202 20075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:45295 every 8 connection(s)
I20260811 05:12:28.221076 19925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:12:28 dist-test-slave-7c35 krb5kdc[19493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425148, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:28.230770 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 19925
I20260811 05:12:28.230973 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:28.231364 31075 external_mini_cluster.cc:1428] Setting key ed68407c471c27ebd3d84e8c17603176
I20260811 05:12:28.234813 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.222875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:37627 (local address 127.30.88.254:35147)
0811 05:12:28.223033 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.223037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.223817 (+ 780us) server_negotiation.cc:407] Connection header received
0811 05:12:28.225063 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.225099 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.225165 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.225267 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:28.226387 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.227047 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.228140 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.228379 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.228473 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.231063 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:28.231067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:28.231072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:28.231103 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:28.232800 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.233414 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.233416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.233418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.233492 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.233910 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.233912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.233913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.234089 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:28.234164 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.234610 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.234705 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:12:28.235590 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.223247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35147 (local address 127.30.88.195:37627)
0811 05:12:28.223667 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.223707 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.224877 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.225438 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.225444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.225773 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:28.226218 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.226231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.227164 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.227167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.228019 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.228029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.228539 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.228542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.228703 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.229291 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:28.229306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:28.230817 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:28.232935 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:28.232940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:28.232950 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:28.233313 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:28.233591 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:28.233594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:28.233596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:28.233786 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:28.234184 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:28.234188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:28.234405 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.234564 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.234793 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":112,"threads_started":1}
I20260811 05:12:28.236624 20076 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35147
I20260811 05:12:28.236840 20076 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:28.237329 20076 heartbeater.cc:511] Master 127.30.88.254:35147 requested a full tablet report, sending...
I20260811 05:12:28.238328 19546 ts_manager.cc:194] Registered new tserver with Master: 2570b567654c4eb08edd4cf8bf07d398 (127.30.88.195:45295)
I20260811 05:12:28.238952 19546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:37627
I20260811 05:12:28.245386 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:28.254545 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.247371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.30.88.254:35147)
0811 05:12:28.247527 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.247530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.247632 (+ 102us) server_negotiation.cc:407] Connection header received
0811 05:12:28.247731 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.247733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.247768 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.247836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:28.248479 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.248968 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.249743 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.249946 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.250025 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.251581 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:28.251584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:28.251586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:28.251609 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:28.253065 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.253513 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.253516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.253517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.253589 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:28.253847 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:28.253849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:28.253850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:28.254026 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:28.254137 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.254346 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.254440 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:12:28.257436 19546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:28.260558 19546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:28.274837 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.269562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:41971 (local address 127.0.0.1:58686)
0811 05:12:28.270074 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.270099 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.270195 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.270739 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.270742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.270761 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:28.270901 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.270911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.271891 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.271897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.273221 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.273233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.274265 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.274269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.274382 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.274418 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.274535 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.274670 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":7,"thread_start_us":101,"threads_started":1}
I20260811 05:12:28.274922 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.268788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36327 (local address 127.0.0.1:41144)
0811 05:12:28.269153 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.269197 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.269326 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.270634 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.270639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.270662 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:28.270789 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.270796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.272296 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.272300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.273444 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.273452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.274507 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.274513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.274607 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.274627 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.274726 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.274781 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":42,"spinlock_wait_cycles":1920,"thread_start_us":141,"threads_started":1}
I20260811 05:12:28.275072 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.269518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.30.88.193:36327)
0811 05:12:28.269917 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.269934 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.269957 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:12:28.270034 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.270051 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.270300 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.270454 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:28.270975 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.272157 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.273561 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.274364 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.274494 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.274627 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.274759 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.274899 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":95,"threads_started":1}
I20260811 05:12:28.275326 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.269847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.30.88.194:41971)
0811 05:12:28.270297 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.270308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.270332 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:12:28.270412 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.270417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.270602 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.270760 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:28.271030 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.271765 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.273362 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.274140 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.274265 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.274417 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.275072 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.275180 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":129,"threads_started":1}
I20260811 05:12:28.275827 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.268879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:45295 (local address 127.0.0.1:41330)
0811 05:12:28.269152 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.269199 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.269326 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.271046 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.271049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.271067 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:28.271153 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.271159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.272562 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.272568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.273763 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.273773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.275423 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.275429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.275538 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.275563 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.275633 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.275698 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":26,"spinlock_wait_cycles":12416,"thread_start_us":94,"threads_started":1}
I20260811 05:12:28.276548 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.269774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.30.88.195:45295)
0811 05:12:28.270562 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.270575 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.270599 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:12:28.270665 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.270670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.270891 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.271299 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:28.271440 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.272431 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.273902 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.274787 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.276148 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.276280 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.276329 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.276417 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":108,"threads_started":1}
I20260811 05:12:28.277944 19842 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 (DEFAULT_TABLE table=test-table [id=554d80ae4f284517a1635c12c017a06b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:28.277936 19689 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 (DEFAULT_TABLE table=test-table [id=554d80ae4f284517a1635c12c017a06b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:28.278960 19842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee6134810bd4e1c9528b133874a93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:28.278965 19689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee6134810bd4e1c9528b133874a93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:28.279063 19998 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 (DEFAULT_TABLE table=test-table [id=554d80ae4f284517a1635c12c017a06b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:28.279953 19998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee6134810bd4e1c9528b133874a93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:28.285204 20097 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Bootstrap starting.
I20260811 05:12:28.286348 20100 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Bootstrap starting.
I20260811 05:12:28.287636 20097 tablet_bootstrap.cc:654] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:28.288435 20100 tablet_bootstrap.cc:654] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:28.289155 20097 log.cc:824] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:28.289460 20100 log.cc:824] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:28.290045 20099 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: Bootstrap starting.
I20260811 05:12:28.291189 20100 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: No bootstrap required, opened a new log
I20260811 05:12:28.291226 20097 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: No bootstrap required, opened a new log
I20260811 05:12:28.291394 20097 ts_tablet_manager.cc:1397] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:28.291399 20100 ts_tablet_manager.cc:1397] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:12:28.292141 20099 tablet_bootstrap.cc:654] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:28.293361 20099 log.cc:824] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:28.294482 20100 raft_consensus.cc:348] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.294762 20100 raft_consensus.cc:374] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:28.294826 20100 raft_consensus.cc:729] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2570b567654c4eb08edd4cf8bf07d398, State: Initialized, Role: FOLLOWER
I20260811 05:12:28.295312 20100 consensus_queue.cc:260] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.295488 20099 tablet_bootstrap.cc:492] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: No bootstrap required, opened a new log
I20260811 05:12:28.295677 20099 ts_tablet_manager.cc:1397] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 05:12:28.295681 20097 raft_consensus.cc:348] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.295928 20097 raft_consensus.cc:374] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:28.295986 20097 raft_consensus.cc:729] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eaa2443e771455d82d73c89ee0f9dab, State: Initialized, Role: FOLLOWER
I20260811 05:12:28.296005 20076 heartbeater.cc:503] Master 127.30.88.254:35147 was elected leader, sending a full tablet report...
I20260811 05:12:28.296381 20100 ts_tablet_manager.cc:1428] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:12:28.296401 20097 consensus_queue.cc:260] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.297071 19920 heartbeater.cc:503] Master 127.30.88.254:35147 was elected leader, sending a full tablet report...
I20260811 05:12:28.297340 20097 ts_tablet_manager.cc:1428] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:28.299881 20099 raft_consensus.cc:348] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.300091 20099 raft_consensus.cc:374] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:28.300175 20099 raft_consensus.cc:729] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd644ff1f11a4119815be5480767ba5c, State: Initialized, Role: FOLLOWER
I20260811 05:12:28.300633 20099 consensus_queue.cc:260] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.301288 19763 heartbeater.cc:503] Master 127.30.88.254:35147 was elected leader, sending a full tablet report...
I20260811 05:12:28.301700 20099 ts_tablet_manager.cc:1428] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:28.373631 20104 raft_consensus.cc:482] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:28.373916 20104 raft_consensus.cc:504] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.375106 20104 leader_election.cc:288] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd644ff1f11a4119815be5480767ba5c (127.30.88.193:36327), 7eaa2443e771455d82d73c89ee0f9dab (127.30.88.194:41971)
I20260811 05:12:28.379119 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.375501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:57887 (local address 127.30.88.193:36327)
0811 05:12:28.375662 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.375669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.375688 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:12:28.375746 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.375749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.375799 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.375872 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:28.376196 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.376773 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.377977 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.378408 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.378561 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.378630 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.378878 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.378971 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:12:28.379177 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.375807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:34589 (local address 127.30.88.194:41971)
0811 05:12:28.376043 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.376049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.376066 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:12:28.376237 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.376241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.376291 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.376378 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:28.376678 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.377165 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.378177 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.378579 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.378719 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.378768 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.378965 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.379045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 05:12:28.379158 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.375351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36327 (local address 127.30.88.195:57887)
0811 05:12:28.375482 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.375497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.375614 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.375930 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.375934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.375954 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:28.376068 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.376073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.376967 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.376972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.377824 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.377836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.378551 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.378555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.378685 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.378715 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.378885 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.378981 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":1}
I20260811 05:12:28.379164 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.375688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:41971 (local address 127.30.88.195:34589)
0811 05:12:28.375985 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.375999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.376113 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.376395 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.376398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.376418 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:28.376558 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.376564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.377271 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.377277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.378051 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.378059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.378719 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.378722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.378822 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.378848 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.378939 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.379002 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":140,"threads_started":1}
I20260811 05:12:28.379907 19714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee6134810bd4e1c9528b133874a93f7" candidate_uuid: "2570b567654c4eb08edd4cf8bf07d398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd644ff1f11a4119815be5480767ba5c" is_pre_election: true
I20260811 05:12:28.379959 19864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee6134810bd4e1c9528b133874a93f7" candidate_uuid: "2570b567654c4eb08edd4cf8bf07d398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eaa2443e771455d82d73c89ee0f9dab" is_pre_election: true
I20260811 05:12:28.380231 19714 raft_consensus.cc:2454] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2570b567654c4eb08edd4cf8bf07d398 in term 0.
I20260811 05:12:28.380287 19864 raft_consensus.cc:2454] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2570b567654c4eb08edd4cf8bf07d398 in term 0.
I20260811 05:12:28.380810 19943 leader_election.cc:302] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2570b567654c4eb08edd4cf8bf07d398, fd644ff1f11a4119815be5480767ba5c; no voters:
I20260811 05:12:28.381105 20104 raft_consensus.cc:2781] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:28.381206 20104 raft_consensus.cc:482] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:28.381275 20104 raft_consensus.cc:3037] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:28.382433 20104 raft_consensus.cc:504] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.382802 20104 leader_election.cc:288] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [CANDIDATE]: Term 1 election: Requested vote from peers fd644ff1f11a4119815be5480767ba5c (127.30.88.193:36327), 7eaa2443e771455d82d73c89ee0f9dab (127.30.88.194:41971)
I20260811 05:12:28.383174 19714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee6134810bd4e1c9528b133874a93f7" candidate_uuid: "2570b567654c4eb08edd4cf8bf07d398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd644ff1f11a4119815be5480767ba5c"
I20260811 05:12:28.383263 19864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee6134810bd4e1c9528b133874a93f7" candidate_uuid: "2570b567654c4eb08edd4cf8bf07d398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eaa2443e771455d82d73c89ee0f9dab"
I20260811 05:12:28.383348 19714 raft_consensus.cc:3037] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:28.383423 19864 raft_consensus.cc:3037] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:28.384418 19714 raft_consensus.cc:2454] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2570b567654c4eb08edd4cf8bf07d398 in term 1.
I20260811 05:12:28.384510 19864 raft_consensus.cc:2454] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2570b567654c4eb08edd4cf8bf07d398 in term 1.
I20260811 05:12:28.384757 19943 leader_election.cc:302] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2570b567654c4eb08edd4cf8bf07d398, fd644ff1f11a4119815be5480767ba5c; no voters:
I20260811 05:12:28.384971 20104 raft_consensus.cc:2781] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:28.385227 20104 raft_consensus.cc:686] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 1 LEADER]: Becoming Leader. State: Replica: 2570b567654c4eb08edd4cf8bf07d398, State: Running, Role: LEADER
I20260811 05:12:28.385576 20104 consensus_queue.cc:237] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } }
I20260811 05:12:28.388824 19543 catalog_manager.cc:5495] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2570b567654c4eb08edd4cf8bf07d398 (127.30.88.195). New cstate: current_term: 1 leader_uuid: "2570b567654c4eb08edd4cf8bf07d398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2570b567654c4eb08edd4cf8bf07d398" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 45295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:28.399370 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.396104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.30.88.195:45295)
0811 05:12:28.396219 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.396222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.396233 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:28.396372 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.396374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.396419 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.396498 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:28.396714 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.397161 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.398021 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.398219 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.398292 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.398591 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:28.399025 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:28.399131 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.399232 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.399263 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:12:28.404838 19714 raft_consensus.cc:1261] T 1ee6134810bd4e1c9528b133874a93f7 P fd644ff1f11a4119815be5480767ba5c [term 1 FOLLOWER]: Refusing update from remote peer 2570b567654c4eb08edd4cf8bf07d398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:28.404846 19864 raft_consensus.cc:1261] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 1 FOLLOWER]: Refusing update from remote peer 2570b567654c4eb08edd4cf8bf07d398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:28.405490 20104 consensus_queue.cc:1035] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eaa2443e771455d82d73c89ee0f9dab" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:28.405750 20108 consensus_queue.cc:1035] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd644ff1f11a4119815be5480767ba5c" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:28.413692 20113 mvcc.cc:205] Tried to move back new op lower bound from 7317197407859437568 to 7317197407789801472. Current Snapshot: MvccSnapshot[applied={T|T < 7317197407859437568}]
I20260811 05:12:28.414397 20115 mvcc.cc:205] Tried to move back new op lower bound from 7317197407859437568 to 7317197407789801472. Current Snapshot: MvccSnapshot[applied={T|T < 7317197407859437568}]
I20260811 05:12:28.422422 19543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:28.422600 19543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:28.425005 19543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ea6dfbb9f634319b1ec33e0ffbcf416: Sending DeleteTablet for 3 replicas of tablet 1ee6134810bd4e1c9528b133874a93f7
I20260811 05:12:28.425746 19998 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:28 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:41330
I20260811 05:12:28.425814 19842 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:28 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:58686
I20260811 05:12:28.426261 20123 tablet_replica.cc:330] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: stopping tablet replica
I20260811 05:12:28.426316 20124 tablet_replica.cc:330] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: stopping tablet replica
I20260811 05:12:28.426285 19689 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee6134810bd4e1c9528b133874a93f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:28 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:41144
I20260811 05:12:28.426622 20124 raft_consensus.cc:2229] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:28.426673 20123 raft_consensus.cc:2229] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:28.426954 20124 raft_consensus.cc:2258] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:28.427248 20123 raft_consensus.cc:2258] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:28.427799 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19615
I20260811 05:12:28.430665 20123 ts_tablet_manager.cc:1905] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:28.433360 20123 ts_tablet_manager.cc:1918] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:28.433449 20123 log.cc:1196] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/1ee6134810bd4e1c9528b133874a93f7
I20260811 05:12:28.433804 20123 ts_tablet_manager.cc:1939] T 1ee6134810bd4e1c9528b133874a93f7 P 2570b567654c4eb08edd4cf8bf07d398: Deleting consensus metadata
I20260811 05:12:28.434994 19531 catalog_manager.cc:4841] TS 2570b567654c4eb08edd4cf8bf07d398 (127.30.88.195:45295): tablet 1ee6134810bd4e1c9528b133874a93f7 (table test-table [id=554d80ae4f284517a1635c12c017a06b]) successfully deleted
I20260811 05:12:28.438313 20124 ts_tablet_manager.cc:1905] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:28.441476 20124 ts_tablet_manager.cc:1918] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:28.441576 20124 log.cc:1196] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/1ee6134810bd4e1c9528b133874a93f7
I20260811 05:12:28.441921 20124 ts_tablet_manager.cc:1939] T 1ee6134810bd4e1c9528b133874a93f7 P 7eaa2443e771455d82d73c89ee0f9dab: Deleting consensus metadata
I20260811 05:12:28.443084 19529 catalog_manager.cc:4841] TS 7eaa2443e771455d82d73c89ee0f9dab (127.30.88.194:41971): tablet 1ee6134810bd4e1c9528b133874a93f7 (table test-table [id=554d80ae4f284517a1635c12c017a06b]) successfully deleted
W20260811 05:12:28.445161 19529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:36327 (error 108)
I20260811 05:12:28.445274 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19769
I20260811 05:12:28.452297 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.451935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36327 (local address 127.0.0.1:41158)
0811 05:12:28.452110 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.452193 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:12:28.452600 19529 catalog_manager.cc:4584] TS fd644ff1f11a4119815be5480767ba5c (127.30.88.193:36327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ee6134810bd4e1c9528b133874a93f7: Network error: Client connection negotiation failed: client connection to 127.30.88.193:36327: connect: Connection refused (error 111)
I20260811 05:12:28.456254 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19925
I20260811 05:12:28.463222 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 19508
2026-08-11T05:12:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2376 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T05:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:28Z Disabled control of system clock
I20260811 05:12:28.496737 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:45975
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:45975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:28.610451 20135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:28.610682 20135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:28.610714 20135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:28.614919 20135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:28.614970 20135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:28.614985 20135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:28.615001 20135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:28.615015 20135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:28.619601 20135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:45975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:28.620514 20135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:28.621726 20135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:28.630028 20142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:28.630095 20143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:28.630041 20145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:28.630254 20135 server_base.cc:1034] running on GCE node
I20260811 05:12:28.630963 20135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:28.632021 20135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:28.633186 20135 hybrid_clock.cc:648] HybridClock initialized: now 1786425148633160 us; error 39 us; skew 500 ppm
I20260811 05:12:28.634958 20135 webserver.cc:460] Webserver started at http://127.30.88.254:36773/ using document root <none> and password file <none>
I20260811 05:12:28.635500 20135 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:28.635562 20135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:28.635766 20135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:28.637727 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "dad92f9bdd59473aa9f887e008db5fae"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "a70c53d7d30aed48da556b387134916b"
server_key_iv: "6d798991afd8b1968f576a590e235f3f"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.638435 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "dad92f9bdd59473aa9f887e008db5fae"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "a70c53d7d30aed48da556b387134916b"
server_key_iv: "6d798991afd8b1968f576a590e235f3f"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.642413 20135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 05:12:28.644711 20153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.645854 20135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:28.645979 20135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "dad92f9bdd59473aa9f887e008db5fae"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "a70c53d7d30aed48da556b387134916b"
server_key_iv: "6d798991afd8b1968f576a590e235f3f"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.646085 20135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:28.668162 20135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:28.668956 20135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:28.669169 20135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:28.676215 20135 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:45975
I20260811 05:12:28.676239 20209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:45975 every 8 connection(s)
I20260811 05:12:28.677107 20135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:28.680253 20211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:28.682433 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20135
I20260811 05:12:28.682562 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:28.682905 31075 external_mini_cluster.cc:1428] Setting key 8d2679fdf920c762f07f41125b1ebb41
I20260811 05:12:28.686389 20211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: Bootstrap starting.
I20260811 05:12:28.688956 20211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:28.689889 20211 log.cc:824] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:28.692181 20211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: No bootstrap required, opened a new log
I20260811 05:12:28.692627 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.684356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.30.88.254:45975)
0811 05:12:28.684960 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.684974 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.685008 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:12:28.685637 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.685653 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.685992 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.686328 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:28.686747 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.687799 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.688733 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.689270 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.689477 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.690599 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:28.690603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:28.690616 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:28.691132 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:28.691202 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:28.691229 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:28.691335 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.691471 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.691725 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":145,"threads_started":1}
I20260811 05:12:28.696034 20211 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } }
I20260811 05:12:28.696229 20211 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:28.696261 20211 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad92f9bdd59473aa9f887e008db5fae, State: Initialized, Role: FOLLOWER
I20260811 05:12:28.696594 20211 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } }
I20260811 05:12:28.696707 20211 raft_consensus.cc:388] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:28.696750 20211 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:28.696815 20211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:28.697773 20211 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } }
I20260811 05:12:28.698060 20211 leader_election.cc:302] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dad92f9bdd59473aa9f887e008db5fae; no voters:
I20260811 05:12:28.698287 20211 leader_election.cc:288] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:28.698372 20217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:28.698639 20217 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [term 1 LEADER]: Becoming Leader. State: Replica: dad92f9bdd59473aa9f887e008db5fae, State: Running, Role: LEADER
I20260811 05:12:28.698969 20217 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } }
I20260811 05:12:28.699468 20211 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:28.700496 20218 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dad92f9bdd59473aa9f887e008db5fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } } }
I20260811 05:12:28.700528 20219 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad92f9bdd59473aa9f887e008db5fae. Latest consensus state: current_term: 1 leader_uuid: "dad92f9bdd59473aa9f887e008db5fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad92f9bdd59473aa9f887e008db5fae" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45975 } } }
I20260811 05:12:28.700706 20218 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:28.700732 20219 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:28.701776 20228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:28.704663 20228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:28.709350 20228 catalog_manager.cc:1347] Generated new cluster ID: 4389670e5ef549d8ab73e5fb16c201a8
I20260811 05:12:28.709414 20228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:28.721758 20228 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:28.722687 20228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:28.730199 20228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: Generated new TSK 0
I20260811 05:12:28.730839 20228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:28.733862 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--builtin_ntp_servers=127.30.88.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:28.845979 20238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:28.846212 20238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:28.846266 20238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:28.850389 20238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:28.850436 20238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:28.850522 20238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:28.855093 20238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:28.856109 20238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:28.857470 20238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:28.867127 20243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:28.867127 20244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:28.867264 20238 server_base.cc:1034] running on GCE node
W20260811 05:12:28.867151 20247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:28.867718 20238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:28.868386 20238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:28.869596 20238 hybrid_clock.cc:648] HybridClock initialized: now 1786425148869553 us; error 58 us; skew 500 ppm
I20260811 05:12:28.871752 20238 webserver.cc:460] Webserver started at http://127.30.88.193:42257/ using document root <none> and password file <none>
I20260811 05:12:28.872259 20238 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:28.872318 20238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:28.872480 20238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:28.874455 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "e88f445248a64b2fa3f1e5f1409e1adb"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "971b5d5962949ea9f12b919c67368d2b"
server_key_iv: "17a5409185dc7519c9e4a350a40a6ade"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.875057 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "e88f445248a64b2fa3f1e5f1409e1adb"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "971b5d5962949ea9f12b919c67368d2b"
server_key_iv: "17a5409185dc7519c9e4a350a40a6ade"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.878510 20238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:28.880796 20253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.881942 20238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:12:28.882071 20238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "e88f445248a64b2fa3f1e5f1409e1adb"
format_stamp: "Formatted at 2026-08-11 05:12:28 on dist-test-slave-7c35"
server_key: "971b5d5962949ea9f12b919c67368d2b"
server_key_iv: "17a5409185dc7519c9e4a350a40a6ade"
server_key_version: "encryptionkey@0"
I20260811 05:12:28.882174 20238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:28.907677 20238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:28.908514 20238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:28.908780 20238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:28.909413 20238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:28.910445 20238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:28.910545 20238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.910640 20238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:28.910677 20238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:28.920181 20238 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:40213
I20260811 05:12:28.920217 20383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:40213 every 8 connection(s)
I20260811 05:12:28.921308 20238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:28.930262 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.923150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:47657 (local address 127.30.88.254:45975)
0811 05:12:28.923306 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:28.923312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:28.924444 (+ 1132us) server_negotiation.cc:407] Connection header received
0811 05:12:28.925787 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:28.925790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:28.925843 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:28.925932 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:28.926757 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.927344 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.928432 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.928652 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.928759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.929819 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:28.929822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:28.929825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:28.929840 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:28.929861 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:28.929870 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:28.929938 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:28.930076 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 05:12:28.930141 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:12:28.931056 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20238
I20260811 05:12:28.931185 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:28.931278 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:28.923617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45975 (local address 127.30.88.193:47657)
0811 05:12:28.924289 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:28.924328 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:28.925571 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:28.926097 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:28.926103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:28.926182 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:28.926626 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.926637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.927461 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.927464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.928321 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:28.928331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:28.928804 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:28.928807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:28.928943 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:28.929562 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:28.929564 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:28.929601 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:28.929661 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:28.929951 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:28.929958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:28.929962 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:28.930061 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 05:12:28.930281 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":134,"threads_started":1}
I20260811 05:12:28.931561 31075 external_mini_cluster.cc:1428] Setting key bd31777348beb483db01bbb64d1ca701
I20260811 05:12:28.932556 20384 heartbeater.cc:347] Connected to a master server at 127.30.88.254:45975
I20260811 05:12:28.932776 20384 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:28.933292 20384 heartbeater.cc:511] Master 127.30.88.254:45975 requested a full tablet report, sending...
I20260811 05:12:28.933597 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--builtin_ntp_servers=127.30.88.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:28.934717 20173 ts_manager.cc:194] Registered new tserver with Master: e88f445248a64b2fa3f1e5f1409e1adb (127.30.88.193:40213)
I20260811 05:12:28.935842 20173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.193:47657
W20260811 05:12:29.047000 20387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:29.047235 20387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:29.047271 20387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:29.051489 20387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:29.051533 20387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:29.051613 20387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:29.056191 20387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:29.057204 20387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:29.058624 20387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:29.068012 20396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:29.068022 20393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:29.068022 20394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:29.068197 20387 server_base.cc:1034] running on GCE node
I20260811 05:12:29.068714 20387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:29.069306 20387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:29.070506 20387 hybrid_clock.cc:648] HybridClock initialized: now 1786425149070476 us; error 39 us; skew 500 ppm
I20260811 05:12:29.072397 20387 webserver.cc:460] Webserver started at http://127.30.88.194:40619/ using document root <none> and password file <none>
I20260811 05:12:29.072962 20387 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:29.073078 20387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:29.073349 20387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:29.075538 20387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "42e31aae9d704154a33469d7d4cb7534"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "601651feea7c5af01f325a20bca73ab5"
server_key_iv: "9d0c864640a19b67f74ecd13a860563b"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.076120 20387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "42e31aae9d704154a33469d7d4cb7534"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "601651feea7c5af01f325a20bca73ab5"
server_key_iv: "9d0c864640a19b67f74ecd13a860563b"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.080035 20387 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:12:29.082530 20402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.083796 20387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:29.083933 20387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "42e31aae9d704154a33469d7d4cb7534"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "601651feea7c5af01f325a20bca73ab5"
server_key_iv: "9d0c864640a19b67f74ecd13a860563b"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.084046 20387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:29.103487 20387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:29.104143 20387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:29.104373 20387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:29.104945 20387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:29.105878 20387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:29.105952 20387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.106027 20387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:29.106082 20387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.115384 20387 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:38155
I20260811 05:12:29.115417 20543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:38155 every 8 connection(s)
I20260811 05:12:29.116217 20387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:29.122228 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20387
I20260811 05:12:29.122354 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:29.122679 31075 external_mini_cluster.cc:1428] Setting key 4a3c7bd4c05670da3518700a968d109f
I20260811 05:12:29.124887 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--builtin_ntp_servers=127.30.88.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:29.125512 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.117905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:38149 (local address 127.30.88.254:45975)
0811 05:12:29.118068 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.118073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.119287 (+ 1214us) server_negotiation.cc:407] Connection header received
0811 05:12:29.120649 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.120652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.120702 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.120789 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:29.121576 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.122074 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.123564 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.123779 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.123867 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.125044 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:29.125050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:29.125054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:29.125071 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:29.125096 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:29.125111 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:29.125170 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.125351 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.125405 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:12:29.126628 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.118440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45975 (local address 127.30.88.194:38149)
0811 05:12:29.119121 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.119175 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.120445 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.120938 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.120944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.121018 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:29.121455 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.121466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.122203 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.122208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.123425 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.123435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.123936 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.123942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.124090 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.124715 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:29.124719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:29.124773 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:29.124847 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:29.125196 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:29.125207 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:29.125216 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.125342 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.125646 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260811 05:12:29.127945 20544 heartbeater.cc:347] Connected to a master server at 127.30.88.254:45975
I20260811 05:12:29.128158 20544 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:29.128630 20544 heartbeater.cc:511] Master 127.30.88.254:45975 requested a full tablet report, sending...
I20260811 05:12:29.129642 20173 ts_manager.cc:194] Registered new tserver with Master: 42e31aae9d704154a33469d7d4cb7534 (127.30.88.194:38155)
I20260811 05:12:29.130286 20173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.194:38149
W20260811 05:12:29.240597 20547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:29.240902 20547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:29.240974 20547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:29.244966 20547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:29.245040 20547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:29.245141 20547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:29.249830 20547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:29.250978 20547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:29.252295 20547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:29.261505 20556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:29.261504 20553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:29.261591 20554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:29.261667 20547 server_base.cc:1034] running on GCE node
I20260811 05:12:29.262116 20547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:29.262625 20547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:29.263799 20547 hybrid_clock.cc:648] HybridClock initialized: now 1786425149263772 us; error 39 us; skew 500 ppm
I20260811 05:12:29.265463 20547 webserver.cc:460] Webserver started at http://127.30.88.195:43721/ using document root <none> and password file <none>
I20260811 05:12:29.265985 20547 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:29.266043 20547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:29.266273 20547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:29.268225 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "e75a7e83118448499a7c252466da2f3b"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "e1a13bfb9b5f0f647c8aa5c040e1304b"
server_key_iv: "f30e041d1af69fee87c8ccef4a08b282"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.268793 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "e75a7e83118448499a7c252466da2f3b"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "e1a13bfb9b5f0f647c8aa5c040e1304b"
server_key_iv: "f30e041d1af69fee87c8ccef4a08b282"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.272384 20547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:29.274712 20563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.275905 20547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:29.276019 20547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "e75a7e83118448499a7c252466da2f3b"
format_stamp: "Formatted at 2026-08-11 05:12:29 on dist-test-slave-7c35"
server_key: "e1a13bfb9b5f0f647c8aa5c040e1304b"
server_key_iv: "f30e041d1af69fee87c8ccef4a08b282"
server_key_version: "encryptionkey@0"
I20260811 05:12:29.276124 20547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:29.298892 20547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:29.299566 20547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:29.299772 20547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:29.300365 20547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:29.301399 20547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:29.301446 20547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.301484 20547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:29.301501 20547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:29.310271 20547 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:32959
I20260811 05:12:29.310302 20695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:32959 every 8 connection(s)
I20260811 05:12:29.311193 20547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:29.312795 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20547
I20260811 05:12:29.312906 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:29.313220 31075 external_mini_cluster.cc:1428] Setting key cb8b11d1b175254e56a08fea6acb1a61
I20260811 05:12:29.320686 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.313004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:42533 (local address 127.30.88.254:45975)
0811 05:12:29.313155 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.313162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.314431 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 05:12:29.316052 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.316055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.316111 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.316196 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:29.317063 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.317619 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.318826 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.319080 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.319199 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.320204 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:29.320208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:29.320211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:29.320223 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:29.320250 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:29.320258 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:29.320332 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.320495 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.320552 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:12:29.321465 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.313456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45975 (local address 127.30.88.195:42533)
0811 05:12:29.314288 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.314335 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.315834 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.316371 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.316378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.316454 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:29.316923 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.316935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.317742 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.317745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.318694 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.318705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.319234 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.319238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.319404 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.319942 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:29.319944 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:29.319999 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:29.320057 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:29.320347 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:29.320358 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:29.320363 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.320491 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.320749 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":177,"threads_started":1}
I20260811 05:12:29.322427 20696 heartbeater.cc:347] Connected to a master server at 127.30.88.254:45975
I20260811 05:12:29.322671 20696 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:29.323196 20696 heartbeater.cc:511] Master 127.30.88.254:45975 requested a full tablet report, sending...
I20260811 05:12:29.324162 20173 ts_manager.cc:194] Registered new tserver with Master: e75a7e83118448499a7c252466da2f3b (127.30.88.195:32959)
I20260811 05:12:29.324738 20173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.195:42533
I20260811 05:12:29.327082 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:29.332561 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.328987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.30.88.254:45975)
0811 05:12:29.329095 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.329098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.329254 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:12:29.329340 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.329342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.329396 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.329459 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:29.329698 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.330182 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.331047 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.331244 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.331318 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.332151 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:29.332154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:29.332155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:29.332167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:29.332185 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:29.332192 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:29.332299 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.332430 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.332463 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:12:29.335467 20173 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:29.338371 20173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:29.353010 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.346767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.30.88.195:32959)
0811 05:12:29.348219 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.348230 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.348253 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:12:29.348324 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.348328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.348495 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.348991 (+ 496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:29.349121 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.350197 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.351472 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352284 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352395 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352597 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.352740 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.352850 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1356,"thread_start_us":193,"threads_started":1}
I20260811 05:12:29.353004 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.346574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:32959 (local address 127.0.0.1:57548)
0811 05:12:29.347240 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.347285 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.347420 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.348646 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.348649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.348674 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:29.348845 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.348852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.350321 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.350324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.351347 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.351358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352416 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.352559 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352599 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.352712 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.352830 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":45,"spinlock_wait_cycles":768,"thread_start_us":78,"threads_started":1}
I20260811 05:12:29.353374 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.347726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.30.88.194:38155)
0811 05:12:29.348149 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.348157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.348176 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:12:29.348237 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.348240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.348371 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.348525 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:29.349033 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.349893 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.351864 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352435 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352538 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352722 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.353100 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.353203 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":130,"threads_started":1}
I20260811 05:12:29.353931 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.347262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59430 (local address 127.30.88.193:40213)
0811 05:12:29.347726 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.347738 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.347856 (+ 118us) server_negotiation.cc:407] Connection header received
0811 05:12:29.348086 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.348093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.348300 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.348491 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:29.348786 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.350102 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.351655 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352402 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352490 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352630 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.353671 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.353765 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":107,"threads_started":1}
I20260811 05:12:29.353061 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.346764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40213 (local address 127.0.0.1:59430)
0811 05:12:29.347748 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.347761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.347949 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.348502 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.348507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.348530 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:29.348658 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.348665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.350226 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.350229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.351502 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.351532 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352674 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.352765 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352788 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.352856 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.352934 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":3,"thread_start_us":198,"threads_started":1}
I20260811 05:12:29.355062 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.347302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:38155 (local address 127.0.0.1:58392)
0811 05:12:29.347882 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.347896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.347980 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.348662 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.348665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.348696 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:29.348919 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.348933 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.350052 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.350056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.351756 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.351763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.352780 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.352783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.352867 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.352887 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.352951 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.353008 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":393,"spinlock_wait_cycles":1664,"thread_start_us":83,"threads_started":1}
I20260811 05:12:29.356454 20618 tablet_service.cc:1467] Processing CreateTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 (DEFAULT_TABLE table=test-table [id=a5a653835f0a471798b4a13b14cb1400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:29.356861 20307 tablet_service.cc:1467] Processing CreateTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 (DEFAULT_TABLE table=test-table [id=a5a653835f0a471798b4a13b14cb1400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:29.357231 20466 tablet_service.cc:1467] Processing CreateTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 (DEFAULT_TABLE table=test-table [id=a5a653835f0a471798b4a13b14cb1400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:29.357695 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb87c3341b4e4e74a926f144cf4f6c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:29.357800 20307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb87c3341b4e4e74a926f144cf4f6c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:29.358234 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb87c3341b4e4e74a926f144cf4f6c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:29.364157 20715 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Bootstrap starting.
I20260811 05:12:29.365447 20716 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Bootstrap starting.
I20260811 05:12:29.366487 20715 tablet_bootstrap.cc:654] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:29.367179 20717 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Bootstrap starting.
I20260811 05:12:29.367236 20716 tablet_bootstrap.cc:654] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:29.367352 20715 log.cc:824] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:29.368017 20716 log.cc:824] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:29.369278 20717 tablet_bootstrap.cc:654] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:29.369506 20715 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: No bootstrap required, opened a new log
I20260811 05:12:29.369705 20715 ts_tablet_manager.cc:1397] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:12:29.369999 20717 log.cc:824] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:29.371737 20717 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: No bootstrap required, opened a new log
I20260811 05:12:29.371914 20717 ts_tablet_manager.cc:1397] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:12:29.373202 20715 raft_consensus.cc:348] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.373463 20715 raft_consensus.cc:374] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:29.373520 20715 raft_consensus.cc:729] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e31aae9d704154a33469d7d4cb7534, State: Initialized, Role: FOLLOWER
I20260811 05:12:29.373950 20715 consensus_queue.cc:260] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.374626 20544 heartbeater.cc:503] Master 127.30.88.254:45975 was elected leader, sending a full tablet report...
I20260811 05:12:29.374967 20715 ts_tablet_manager.cc:1428] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 05:12:29.375212 20717 raft_consensus.cc:348] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.375458 20717 raft_consensus.cc:374] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:29.375515 20717 raft_consensus.cc:729] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75a7e83118448499a7c252466da2f3b, State: Initialized, Role: FOLLOWER
I20260811 05:12:29.375948 20717 consensus_queue.cc:260] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.376837 20717 ts_tablet_manager.cc:1428] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:12:29.376912 20716 tablet_bootstrap.cc:492] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: No bootstrap required, opened a new log
I20260811 05:12:29.377134 20716 ts_tablet_manager.cc:1397] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260811 05:12:29.377204 20696 heartbeater.cc:503] Master 127.30.88.254:45975 was elected leader, sending a full tablet report...
I20260811 05:12:29.381165 20716 raft_consensus.cc:348] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.381389 20716 raft_consensus.cc:374] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:29.381455 20716 raft_consensus.cc:729] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e88f445248a64b2fa3f1e5f1409e1adb, State: Initialized, Role: FOLLOWER
I20260811 05:12:29.381803 20716 consensus_queue.cc:260] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.382383 20384 heartbeater.cc:503] Master 127.30.88.254:45975 was elected leader, sending a full tablet report...
I20260811 05:12:29.382592 20716 ts_tablet_manager.cc:1428] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 05:12:29.422433 20385 tablet.cc:2369] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:29.445355 20724 raft_consensus.cc:482] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:29.445600 20724 raft_consensus.cc:504] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.446815 20724 leader_election.cc:288] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e88f445248a64b2fa3f1e5f1409e1adb (127.30.88.193:40213), 42e31aae9d704154a33469d7d4cb7534 (127.30.88.194:38155)
I20260811 05:12:29.451287 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.447117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40213 (local address 127.30.88.195:39325)
0811 05:12:29.447268 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.447288 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.447425 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.447788 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.447790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.447808 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:29.447976 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.447984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.449018 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.449021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.449987 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.449996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.450843 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.450864 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.450977 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.451014 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.451088 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.451172 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 05:12:29.451380 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.447237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:39325 (local address 127.30.88.193:40213)
0811 05:12:29.447389 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.447394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.447413 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:12:29.447546 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.447549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.447634 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.447721 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:29.448110 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.448882 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.450121 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.450722 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.450838 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.450949 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.451121 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.451204 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:12:29.451586 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.447568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:35143 (local address 127.30.88.194:38155)
0811 05:12:29.447728 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.447731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.447877 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:12:29.448007 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.448013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.448089 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.448158 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:29.448571 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.449180 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.450463 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.450932 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.451060 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.451116 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.451356 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.451447 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:12:29.451964 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.447389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:38155 (local address 127.30.88.195:35143)
0811 05:12:29.447776 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.447792 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:29.447904 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:29.448201 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:29.448204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:29.448224 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:29.448403 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.448412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.449288 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.449291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.450309 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:29.450321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.451091 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.451098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:29.451210 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.451235 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:12:29.451324 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:12:29.451817 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":1,"thread_start_us":158,"threads_started":1}
I20260811 05:12:29.452032 20329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb87c3341b4e4e74a926f144cf4f6c01" candidate_uuid: "e75a7e83118448499a7c252466da2f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" is_pre_election: true
I20260811 05:12:29.452397 20329 raft_consensus.cc:2454] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75a7e83118448499a7c252466da2f3b in term 0.
I20260811 05:12:29.452741 20489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb87c3341b4e4e74a926f144cf4f6c01" candidate_uuid: "e75a7e83118448499a7c252466da2f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e31aae9d704154a33469d7d4cb7534" is_pre_election: true
I20260811 05:12:29.452991 20489 raft_consensus.cc:2454] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75a7e83118448499a7c252466da2f3b in term 0.
I20260811 05:12:29.452970 20567 leader_election.cc:302] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e75a7e83118448499a7c252466da2f3b, e88f445248a64b2fa3f1e5f1409e1adb; no voters:
I20260811 05:12:29.453409 20724 raft_consensus.cc:2781] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:29.453514 20724 raft_consensus.cc:482] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:29.453567 20724 raft_consensus.cc:3037] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:29.454670 20724 raft_consensus.cc:504] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.455070 20724 leader_election.cc:288] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [CANDIDATE]: Term 1 election: Requested vote from peers e88f445248a64b2fa3f1e5f1409e1adb (127.30.88.193:40213), 42e31aae9d704154a33469d7d4cb7534 (127.30.88.194:38155)
I20260811 05:12:29.455374 20329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb87c3341b4e4e74a926f144cf4f6c01" candidate_uuid: "e75a7e83118448499a7c252466da2f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e88f445248a64b2fa3f1e5f1409e1adb"
I20260811 05:12:29.455495 20329 raft_consensus.cc:3037] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:29.455488 20489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb87c3341b4e4e74a926f144cf4f6c01" candidate_uuid: "e75a7e83118448499a7c252466da2f3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e31aae9d704154a33469d7d4cb7534"
I20260811 05:12:29.455646 20489 raft_consensus.cc:3037] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:29.456530 20329 raft_consensus.cc:2454] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75a7e83118448499a7c252466da2f3b in term 1.
I20260811 05:12:29.456743 20489 raft_consensus.cc:2454] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75a7e83118448499a7c252466da2f3b in term 1.
I20260811 05:12:29.456873 20567 leader_election.cc:302] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e75a7e83118448499a7c252466da2f3b, e88f445248a64b2fa3f1e5f1409e1adb; no voters:
I20260811 05:12:29.457244 20724 raft_consensus.cc:2781] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:29.457649 20724 raft_consensus.cc:686] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 1 LEADER]: Becoming Leader. State: Replica: e75a7e83118448499a7c252466da2f3b, State: Running, Role: LEADER
I20260811 05:12:29.458047 20724 consensus_queue.cc:237] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } }
I20260811 05:12:29.461161 20172 catalog_manager.cc:5495] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b reported cstate change: term changed from 0 to 1, leader changed from <none> to e75a7e83118448499a7c252466da2f3b (127.30.88.195). New cstate: current_term: 1 leader_uuid: "e75a7e83118448499a7c252466da2f3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e75a7e83118448499a7c252466da2f3b" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 32959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:29.475848 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.472228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.30.88.195:32959)
0811 05:12:29.472410 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:29.472415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:29.472431 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:12:29.472522 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:29.472525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:29.472579 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:29.472700 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:29.472973 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.473490 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.474334 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:29.474549 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:29.474653 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:29.475005 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:29.475398 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:29.475553 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:29.475667 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 05:12:29.475703 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:12:29.482072 20489 raft_consensus.cc:1261] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 1 FOLLOWER]: Refusing update from remote peer e75a7e83118448499a7c252466da2f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:29.482072 20329 raft_consensus.cc:1261] T eb87c3341b4e4e74a926f144cf4f6c01 P e88f445248a64b2fa3f1e5f1409e1adb [term 1 FOLLOWER]: Refusing update from remote peer e75a7e83118448499a7c252466da2f3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:29.482810 20724 consensus_queue.cc:1035] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "42e31aae9d704154a33469d7d4cb7534" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:29.483076 20727 consensus_queue.cc:1035] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e88f445248a64b2fa3f1e5f1409e1adb" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 40213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:29.491851 20732 mvcc.cc:205] Tried to move back new op lower bound from 7317197412270317568 to 7317197412182880256. Current Snapshot: MvccSnapshot[applied={T|T < 7317197412270317568}]
I20260811 05:12:29.492090 20731 mvcc.cc:205] Tried to move back new op lower bound from 7317197412270317568 to 7317197412182880256. Current Snapshot: MvccSnapshot[applied={T|T < 7317197412270317568}]
I20260811 05:12:29.499966 20172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:29.500221 20172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:29.502803 20172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dad92f9bdd59473aa9f887e008db5fae: Sending DeleteTablet for 3 replicas of tablet eb87c3341b4e4e74a926f144cf4f6c01
I20260811 05:12:29.503645 20618 tablet_service.cc:1514] Processing DeleteTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:29 UTC) from {username='slave'} at 127.0.0.1:57548
I20260811 05:12:29.503906 20307 tablet_service.cc:1514] Processing DeleteTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:29 UTC) from {username='slave'} at 127.0.0.1:59430
I20260811 05:12:29.504266 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20238
I20260811 05:12:29.504410 20466 tablet_service.cc:1514] Processing DeleteTablet for tablet eb87c3341b4e4e74a926f144cf4f6c01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:29 UTC) from {username='slave'} at 127.0.0.1:58392
I20260811 05:12:29.508701 20742 tablet_replica.cc:330] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: stopping tablet replica
I20260811 05:12:29.509053 20742 raft_consensus.cc:2229] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:29.509403 20742 raft_consensus.cc:2258] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:29.510962 20742 ts_tablet_manager.cc:1905] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:29.511181 20740 tablet_replica.cc:330] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: stopping tablet replica
I20260811 05:12:29.511546 20740 raft_consensus.cc:2229] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:29.512061 20740 raft_consensus.cc:2258] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:29.513489 20740 ts_tablet_manager.cc:1905] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:29.514102 20742 ts_tablet_manager.cc:1918] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:29.514218 20742 log.cc:1196] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/eb87c3341b4e4e74a926f144cf4f6c01
I20260811 05:12:29.514572 20742 ts_tablet_manager.cc:1939] T eb87c3341b4e4e74a926f144cf4f6c01 P 42e31aae9d704154a33469d7d4cb7534: Deleting consensus metadata
W20260811 05:12:29.515916 20157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:40213 (error 108)
I20260811 05:12:29.516433 20157 catalog_manager.cc:4841] TS 42e31aae9d704154a33469d7d4cb7534 (127.30.88.194:38155): tablet eb87c3341b4e4e74a926f144cf4f6c01 (table test-table [id=a5a653835f0a471798b4a13b14cb1400]) successfully deleted
I20260811 05:12:29.516738 20740 ts_tablet_manager.cc:1918] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:29.516868 20740 log.cc:1196] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/eb87c3341b4e4e74a926f144cf4f6c01
I20260811 05:12:29.517143 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:29.516856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:40213 (local address 127.0.0.1:59434)
0811 05:12:29.516971 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:29.517031 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:12:29.517239 20740 ts_tablet_manager.cc:1939] T eb87c3341b4e4e74a926f144cf4f6c01 P e75a7e83118448499a7c252466da2f3b: Deleting consensus metadata
W20260811 05:12:29.517302 20157 catalog_manager.cc:4584] TS e88f445248a64b2fa3f1e5f1409e1adb (127.30.88.193:40213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb87c3341b4e4e74a926f144cf4f6c01: Network error: Client connection negotiation failed: client connection to 127.30.88.193:40213: connect: Connection refused (error 111)
I20260811 05:12:29.517776 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20387
I20260811 05:12:29.519590 20154 catalog_manager.cc:4841] TS e75a7e83118448499a7c252466da2f3b (127.30.88.195:32959): tablet eb87c3341b4e4e74a926f144cf4f6c01 (table test-table [id=a5a653835f0a471798b4a13b14cb1400]) successfully deleted
I20260811 05:12:29.526932 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20547
I20260811 05:12:29.533349 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20135
2026-08-11T05:12:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1069 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:29 dist-test-slave-7c35 krb5kdc[20746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:29 dist-test-slave-7c35 krb5kdc[20746](info): set up 2 sockets
Aug 11 05:12:29 dist-test-slave-7c35 krb5kdc[20746](info): commencing operation
krb5kdc: starting...
W20260811 05:12:30.638154 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:30 dist-test-slave-7c35 krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:30.761475 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39765
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:39765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:30.873847 20762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:30.874150 20762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:30.874191 20762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:30.878213 20762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:30.878273 20762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:30.878293 20762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:30.878309 20762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:30.878324 20762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:30.883607 20762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:39765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:39765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:30.884672 20762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:30.885673 20762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:30.894330 20768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:30.894363 20769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:30.894313 20772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:30.894619 20762 server_base.cc:1034] running on GCE node
I20260811 05:12:30.895287 20762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:30.896456 20762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:30.897660 20762 hybrid_clock.cc:648] HybridClock initialized: now 1786425150897641 us; error 35 us; skew 500 ppm
Aug 11 05:12:30 dist-test-slave-7c35 krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:30.900535 20762 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:30.901454 20762 webserver.cc:460] Webserver started at http://127.30.88.254:44647/ using document root <none> and password file <none>
I20260811 05:12:30.902009 20762 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:30.902069 20762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:30.902238 20762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:30.904328 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "99be0a149f314340b6515d63cda848d1"
format_stamp: "Formatted at 2026-08-11 05:12:30 on dist-test-slave-7c35"
server_key: "88a9fe2ca25a1fb7f46266fe24812219"
server_key_iv: "e57782a5e85cd71a7693e41153ffbdaa"
server_key_version: "encryptionkey@0"
I20260811 05:12:30.904844 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "99be0a149f314340b6515d63cda848d1"
format_stamp: "Formatted at 2026-08-11 05:12:30 on dist-test-slave-7c35"
server_key: "88a9fe2ca25a1fb7f46266fe24812219"
server_key_iv: "e57782a5e85cd71a7693e41153ffbdaa"
server_key_version: "encryptionkey@0"
I20260811 05:12:30.908694 20762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:12:30.911459 20779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:30.912853 20762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:30.912983 20762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "99be0a149f314340b6515d63cda848d1"
format_stamp: "Formatted at 2026-08-11 05:12:30 on dist-test-slave-7c35"
server_key: "88a9fe2ca25a1fb7f46266fe24812219"
server_key_iv: "e57782a5e85cd71a7693e41153ffbdaa"
server_key_version: "encryptionkey@0"
I20260811 05:12:30.913095 20762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:30.934325 20762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:30.937672 20762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:30.937867 20762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:30.945998 20762 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:39765
I20260811 05:12:30.946010 20838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:39765 every 8 connection(s)
I20260811 05:12:30.947156 20762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:30.948653 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20762
I20260811 05:12:30.948815 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:30.949142 31075 external_mini_cluster.cc:1428] Setting key a283d4068870359dde484cd40eab0833
I20260811 05:12:30.950598 20839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:30.957535 20839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: Bootstrap starting.
Aug 11 05:12:30 dist-test-slave-7c35 krb5kdc[20746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:30.960281 20839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:30.961009 20839 log.cc:824] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:30.963418 20839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: No bootstrap required, opened a new log
I20260811 05:12:30.965663 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:30.950759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.30.88.254:39765)
0811 05:12:30.951296 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:30.951313 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:30.951354 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:12:30.952061 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:30.952078 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:30.952456 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:30.952760 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:30.953737 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:30.954937 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:30.956120 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:30.956704 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:30.956837 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:30.959926 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:30.959931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:30.959947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:30.959978 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:30.962112 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:30.962632 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:30.962638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:30.962644 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:30.962777 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:30.963119 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:30.963122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:30.963124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:30.963657 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:30.963862 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:30.964108 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 05:12:30.964412 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":147,"threads_started":1}
I20260811 05:12:30.967425 20839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } }
I20260811 05:12:30.967644 20839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:30.967707 20839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99be0a149f314340b6515d63cda848d1, State: Initialized, Role: FOLLOWER
I20260811 05:12:30.968194 20839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } }
I20260811 05:12:30.968365 20839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:30.968432 20839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:30.968559 20839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:30.969515 20839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } }
I20260811 05:12:30.969874 20839 leader_election.cc:302] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99be0a149f314340b6515d63cda848d1; no voters:
I20260811 05:12:30.970227 20839 leader_election.cc:288] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:30.970419 20844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:30.970772 20844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [term 1 LEADER]: Becoming Leader. State: Replica: 99be0a149f314340b6515d63cda848d1, State: Running, Role: LEADER
I20260811 05:12:30.971146 20844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } }
I20260811 05:12:30.971674 20839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:30.972810 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99be0a149f314340b6515d63cda848d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } } }
I20260811 05:12:30.972967 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:30.973407 20855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:30.973354 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99be0a149f314340b6515d63cda848d1. Latest consensus state: current_term: 1 leader_uuid: "99be0a149f314340b6515d63cda848d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99be0a149f314340b6515d63cda848d1" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 39765 } } }
I20260811 05:12:30.973518 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:30.976656 20855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:30.983019 20855 catalog_manager.cc:1347] Generated new cluster ID: aa15085e15ac454ab5a9a9051b5a799c
I20260811 05:12:30.983114 20855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:31.009081 20855 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:31.010188 20855 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:31.015794 20855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: Generated new TSK 0
I20260811 05:12:31.016624 20855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.88.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
Entry for principal kudu/127.30.88.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab.
I20260811 05:12:31.055106 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:31.180230 20867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:31.180540 20867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:31.180589 20867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:31.184813 20867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:31.184881 20867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:31.184944 20867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:31.190235 20867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.193.keytab
--principal=kudu/127.30.88.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:31.191262 20867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:31.192358 20867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:31.202040 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:31.202042 20876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:31.202098 20867 server_base.cc:1034] running on GCE node
W20260811 05:12:31.202430 20874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:31.202801 20867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:31.203454 20867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:31.204625 20867 hybrid_clock.cc:648] HybridClock initialized: now 1786425151204609 us; error 29 us; skew 500 ppm
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:31.207257 20867 init.cc:377] Logged in from keytab as kudu/127.30.88.193@KRBTEST.COM (short username kudu)
I20260811 05:12:31.208055 20867 webserver.cc:460] Webserver started at http://127.30.88.193:44381/ using document root <none> and password file <none>
I20260811 05:12:31.208618 20867 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:31.208678 20867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:31.208900 20867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:31.210904 20867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "51bb1a3a91ac4cc1b2093a8dad512145"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "f0f79578a9e3e637883a05c401bcb6ee"
server_key_iv: "bc1c7a67dfd024a06c01307a760438ad"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.211494 20867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "51bb1a3a91ac4cc1b2093a8dad512145"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "f0f79578a9e3e637883a05c401bcb6ee"
server_key_iv: "bc1c7a67dfd024a06c01307a760438ad"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.215000 20867 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:31.217238 20886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.218638 20867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:31.218765 20867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "51bb1a3a91ac4cc1b2093a8dad512145"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "f0f79578a9e3e637883a05c401bcb6ee"
server_key_iv: "bc1c7a67dfd024a06c01307a760438ad"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.218901 20867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:31.229876 20867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:31.232637 20867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:31.232827 20867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:31.233373 20867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:31.234174 20867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:31.234251 20867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.234336 20867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:31.234418 20867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.243391 20867 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:36053
I20260811 05:12:31.243422 21024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:36053 every 8 connection(s)
I20260811 05:12:31.244256 20867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:31.252753 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 20867
I20260811 05:12:31.252908 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:31.253305 31075 external_mini_cluster.cc:1428] Setting key daddbf5283c9cc1da2102fee2b969cc4
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.193@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:31.258548 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.245972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:51945 (local address 127.30.88.254:39765)
0811 05:12:31.246111 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.246114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.246918 (+ 804us) server_negotiation.cc:407] Connection header received
0811 05:12:31.248285 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.248289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.248347 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.248457 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:31.249609 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.250171 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.251344 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.251598 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.251713 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.254454 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:31.254457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:31.254460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:31.254482 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:31.256292 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.256894 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.256900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.256904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.256998 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.257417 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.257420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.257422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.257606 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:31.257722 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.258233 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.258374 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:12:31.259392 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.246320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39765 (local address 127.30.88.193:51945)
0811 05:12:31.246735 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.246788 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.248078 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.248629 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.248635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.248990 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:31.249447 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.249460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.250310 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.250318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.251214 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.251223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.251753 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.251757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.251924 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.252463 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:31.252478 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:31.254292 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:31.256412 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.256420 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.256435 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.256791 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.257103 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.257109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.257112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.257281 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.257715 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:31.257722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:31.257995 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.258187 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.258499 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
I20260811 05:12:31.260694 21025 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39765
I20260811 05:12:31.260958 21025 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:31.261561 21025 heartbeater.cc:511] Master 127.30.88.254:39765 requested a full tablet report, sending...
I20260811 05:12:31.263185 20798 ts_manager.cc:194] Registered new tserver with Master: 51bb1a3a91ac4cc1b2093a8dad512145 (127.30.88.193:36053)
WARNING: no policy specified for kudu/127.30.88.194@KRBTEST.COM; defaulting to no policy
I20260811 05:12:31.264441 20798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.193@KRBTEST.COM'} at 127.30.88.193:51945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
Entry for principal kudu/127.30.88.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab.
I20260811 05:12:31.282495 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:31.397578 21031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:31.397850 21031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:31.397938 21031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:31.401886 21031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:31.401948 21031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:31.402007 21031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:31.407224 21031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.194.keytab
--principal=kudu/127.30.88.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:31.408283 21031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:31.409356 21031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:31.418780 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:31.418910 21037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:31.418756 21039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:31.419216 21031 server_base.cc:1034] running on GCE node
I20260811 05:12:31.419634 21031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:31.420280 21031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:31.421497 21031 hybrid_clock.cc:648] HybridClock initialized: now 1786425151421460 us; error 53 us; skew 500 ppm
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:31.424490 21031 init.cc:377] Logged in from keytab as kudu/127.30.88.194@KRBTEST.COM (short username kudu)
I20260811 05:12:31.425372 21031 webserver.cc:460] Webserver started at http://127.30.88.194:42635/ using document root <none> and password file <none>
I20260811 05:12:31.425961 21031 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:31.426060 21031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:31.426312 21031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:31.428535 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "f44f341ce36a4d9590862db401ebe4cd"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "a68562a75b2c5b84de979ed94a632e62"
server_key_iv: "dddffc29ece4b0592aa6879cfbfbc2f8"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.429152 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "f44f341ce36a4d9590862db401ebe4cd"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "a68562a75b2c5b84de979ed94a632e62"
server_key_iv: "dddffc29ece4b0592aa6879cfbfbc2f8"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.432991 21031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:12:31.435400 21047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.436471 21031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:31.436605 21031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "f44f341ce36a4d9590862db401ebe4cd"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "a68562a75b2c5b84de979ed94a632e62"
server_key_iv: "dddffc29ece4b0592aa6879cfbfbc2f8"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.436713 21031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:31.453058 21031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:31.456529 21031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:31.456753 21031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:31.457372 21031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:31.458252 21031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:31.458330 21031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.458407 21031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:31.458468 21031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.467856 21031 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:33005
I20260811 05:12:31.467871 21180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:33005 every 8 connection(s)
I20260811 05:12:31.468804 21031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.194@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:31.478982 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21031
I20260811 05:12:31.479120 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:31.479457 31075 external_mini_cluster.cc:1428] Setting key 8caf488d710671aef4bdb4f360490448
I20260811 05:12:31.482795 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.470521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:48407 (local address 127.30.88.254:39765)
0811 05:12:31.470701 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.470706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.471588 (+ 882us) server_negotiation.cc:407] Connection header received
0811 05:12:31.473160 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.473163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.473213 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.473372 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:31.474495 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.475091 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.476259 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.476483 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.476600 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.479108 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:31.479114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:31.479120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:31.479146 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:31.480639 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.481253 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.481256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.481257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.481331 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.481736 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.481738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.481739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.481898 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:31.481971 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.482580 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.482683 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 05:12:31.483817 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.470979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39765 (local address 127.30.88.194:48407)
0811 05:12:31.471419 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.471457 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.472933 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.473512 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.473519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.473840 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:31.474350 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.474363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.475217 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.475225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.476132 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.476142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.476636 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.476640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.476799 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.477319 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:31.477334 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:31.478907 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:31.480779 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.480789 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.480799 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.481149 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.481434 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.481440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.481445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.481640 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.482004 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:31.482013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:31.482322 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.482544 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.482950 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260811 05:12:31.484910 21181 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39765
I20260811 05:12:31.485169 21181 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:31.485637 21181 heartbeater.cc:511] Master 127.30.88.254:39765 requested a full tablet report, sending...
I20260811 05:12:31.486672 20798 ts_manager.cc:194] Registered new tserver with Master: f44f341ce36a4d9590862db401ebe4cd (127.30.88.194:33005)
I20260811 05:12:31.487270 20798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.194@KRBTEST.COM'} at 127.30.88.194:48407
WARNING: no policy specified for kudu/127.30.88.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
Entry for principal kudu/127.30.88.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab.
I20260811 05:12:31.509719 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:31.624598 21188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:31.624907 21188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:31.624977 21188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:31.629251 21188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:31.629365 21188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:31.629475 21188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:31.634999 21188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.195.keytab
--principal=kudu/127.30.88.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:39765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:31.636061 21188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:31.637035 21188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:31.646628 21197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:31.646646 21193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:31.646636 21195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:31.646982 21188 server_base.cc:1034] running on GCE node
I20260811 05:12:31.647441 21188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:31.648059 21188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:31.649250 21188 hybrid_clock.cc:648] HybridClock initialized: now 1786425151649221 us; error 49 us; skew 500 ppm
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:31.657610 21188 init.cc:377] Logged in from keytab as kudu/127.30.88.195@KRBTEST.COM (short username kudu)
I20260811 05:12:31.658535 21188 webserver.cc:460] Webserver started at http://127.30.88.195:33723/ using document root <none> and password file <none>
I20260811 05:12:31.659194 21188 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:31.659287 21188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:31.659526 21188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:31.661556 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "f71377f7ff1c4b68a0fb47673a3eaafe"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "1ff027a203edda16aec4c420d7848b90"
server_key_iv: "9002a8c48c787fd2a74ac9b012e43e61"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.662115 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "f71377f7ff1c4b68a0fb47673a3eaafe"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "1ff027a203edda16aec4c420d7848b90"
server_key_iv: "9002a8c48c787fd2a74ac9b012e43e61"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.665836 21188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:12:31.668246 21204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.669481 21188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:31.669620 21188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "f71377f7ff1c4b68a0fb47673a3eaafe"
format_stamp: "Formatted at 2026-08-11 05:12:31 on dist-test-slave-7c35"
server_key: "1ff027a203edda16aec4c420d7848b90"
server_key_iv: "9002a8c48c787fd2a74ac9b012e43e61"
server_key_version: "encryptionkey@0"
I20260811 05:12:31.669730 21188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:31.697944 21188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:31.702301 21188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:31.702559 21188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:31.703342 21188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:31.704375 21188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:31.704433 21188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.704491 21188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:31.704522 21188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:31.715770 21188 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:33133
I20260811 05:12:31.715855 21337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:33133 every 8 connection(s)
I20260811 05:12:31.716777 21188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:31.717612 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21188
I20260811 05:12:31.717752 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:31.718091 31075 external_mini_cluster.cc:1428] Setting key 35da0d8829c7f03c84eeee0afdaea1ba
Aug 11 05:12:31 dist-test-slave-7c35 krb5kdc[20746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.195@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:31.731165 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.719075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:50937 (local address 127.30.88.254:39765)
0811 05:12:31.719208 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.719213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.720250 (+ 1037us) server_negotiation.cc:407] Connection header received
0811 05:12:31.721613 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.721616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.721658 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.721750 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:31.723043 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.723642 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.724920 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.725144 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.725251 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.727764 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:31.727767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:31.727770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:31.727793 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:31.728999 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.729506 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.729532 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.729533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.729605 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.729943 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.729946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.729948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.730146 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:31.730265 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.730825 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.730993 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:12:31.731932 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.719549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:39765 (local address 127.30.88.195:50937)
0811 05:12:31.720098 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.720151 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.721429 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.721945 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.721952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.722343 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:31.722849 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.722890 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.723762 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.723766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.724799 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.724809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.725285 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.725289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.725484 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.726045 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:31.726061 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:31.727605 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:31.729137 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.729142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.729155 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.729398 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.729697 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:31.729701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:31.729703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:31.729841 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:31.730262 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:31.730269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:31.730502 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.730700 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.731075 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260811 05:12:31.733069 21338 heartbeater.cc:347] Connected to a master server at 127.30.88.254:39765
I20260811 05:12:31.733315 21338 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:31.733799 21338 heartbeater.cc:511] Master 127.30.88.254:39765 requested a full tablet report, sending...
I20260811 05:12:31.734828 20798 ts_manager.cc:194] Registered new tserver with Master: f71377f7ff1c4b68a0fb47673a3eaafe (127.30.88.195:33133)
I20260811 05:12:31.735473 20798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.88.195@KRBTEST.COM'} at 127.30.88.195:50937
I20260811 05:12:31.743746 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:31.752441 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.745641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.30.88.254:39765)
0811 05:12:31.745772 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.745776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.745963 (+ 187us) server_negotiation.cc:407] Connection header received
0811 05:12:31.746053 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.746055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.746089 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.746149 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:31.746901 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.747388 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.748183 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.748376 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.748446 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.749730 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:31.749733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:31.749735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:31.749754 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:31.751074 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.751461 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.751463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.751465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.751531 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:31.751756 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:31.751758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:31.751759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:31.751939 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:31.752034 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.752194 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.752275 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:12:31.755064 20798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:31.758126 20798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:31.773016 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.767169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.30.88.193:36053)
0811 05:12:31.767567 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.767579 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.767625 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:12:31.768670 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.768679 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.768872 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.769037 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:31.769289 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.770298 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.771788 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772361 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.772487 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.772741 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.772795 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.772881 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":92,"threads_started":1}
I20260811 05:12:31.773002 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.766288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36053 (local address 127.0.0.1:40940)
0811 05:12:31.767540 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.767555 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.767663 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.769022 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.769025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.769058 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:31.769161 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.769167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.770416 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.770422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.771673 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.771684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772488 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.772492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.772607 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.772653 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.772751 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.772865 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1194,"mutex_wait_us":32,"thread_start_us":102,"threads_started":1}
I20260811 05:12:31.773134 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.766198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:33133 (local address 127.0.0.1:60604)
0811 05:12:31.767460 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.767509 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.767640 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.768476 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.768482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.768504 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:31.768621 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.768628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.770051 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.770055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.771584 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.771595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772701 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.772704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.772809 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.772840 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.772890 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.772954 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":75,"thread_start_us":83,"threads_started":1}
I20260811 05:12:31.773470 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.767986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.30.88.194:33005)
0811 05:12:31.768291 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.768302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.768325 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:12:31.768399 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.768404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.768560 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.768717 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:31.768907 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.769776 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.771728 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772566 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.772678 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.772808 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.773208 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.773318 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":125,"threads_started":1}
I20260811 05:12:31.773782 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.767592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:33005 (local address 127.0.0.1:40312)
0811 05:12:31.767848 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.767861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.767939 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.768691 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.768694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.768711 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:31.768809 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.768815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.769903 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.769908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.771583 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.771595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772895 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.772899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.772973 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.772993 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.773067 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.773123 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":125,"thread_start_us":103,"threads_started":1}
I20260811 05:12:31.773940 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.766451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.30.88.195:33133)
0811 05:12:31.767970 (+ 1519us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.767984 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.768009 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:12:31.768084 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.768088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.768306 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.768468 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:31.768752 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.769911 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.771738 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.772579 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.773471 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.773640 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.773707 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.773791 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1406,"thread_start_us":176,"threads_started":1}
I20260811 05:12:31.776345 20941 tablet_service.cc:1467] Processing CreateTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 (DEFAULT_TABLE table=test-table [id=2ba70a2977fa4d56873bc8f1b22c50e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:31.776434 21104 tablet_service.cc:1467] Processing CreateTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 (DEFAULT_TABLE table=test-table [id=2ba70a2977fa4d56873bc8f1b22c50e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:31.776433 21260 tablet_service.cc:1467] Processing CreateTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 (DEFAULT_TABLE table=test-table [id=2ba70a2977fa4d56873bc8f1b22c50e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:31.777407 20941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776f1ff2d44c436c83c37fcf1d491ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:31.777451 21104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776f1ff2d44c436c83c37fcf1d491ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:31.777451 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776f1ff2d44c436c83c37fcf1d491ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:31.782965 21358 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: Bootstrap starting.
I20260811 05:12:31.784897 21360 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Bootstrap starting.
I20260811 05:12:31.785264 21358 tablet_bootstrap.cc:654] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:31.785948 21358 log.cc:824] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:31.786584 21360 tablet_bootstrap.cc:654] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:31.787231 21359 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Bootstrap starting.
I20260811 05:12:31.787927 21358 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: No bootstrap required, opened a new log
I20260811 05:12:31.787927 21360 log.cc:824] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:31.788193 21358 ts_tablet_manager.cc:1397] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 05:12:31.789112 21359 tablet_bootstrap.cc:654] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:31.790004 21359 log.cc:824] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:31.791735 21359 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: No bootstrap required, opened a new log
I20260811 05:12:31.791909 21359 ts_tablet_manager.cc:1397] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:12:31.792040 21358 raft_consensus.cc:348] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.792326 21358 raft_consensus.cc:374] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:31.792438 21358 raft_consensus.cc:729] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb1a3a91ac4cc1b2093a8dad512145, State: Initialized, Role: FOLLOWER
I20260811 05:12:31.792665 21360 tablet_bootstrap.cc:492] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: No bootstrap required, opened a new log
I20260811 05:12:31.792876 21360 ts_tablet_manager.cc:1397] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 05:12:31.792900 21358 consensus_queue.cc:260] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.793660 21025 heartbeater.cc:503] Master 127.30.88.254:39765 was elected leader, sending a full tablet report...
I20260811 05:12:31.793967 21358 ts_tablet_manager.cc:1428] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:12:31.795423 21359 raft_consensus.cc:348] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.795668 21359 raft_consensus.cc:374] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:31.795779 21359 raft_consensus.cc:729] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44f341ce36a4d9590862db401ebe4cd, State: Initialized, Role: FOLLOWER
I20260811 05:12:31.796271 21359 consensus_queue.cc:260] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.796281 21360 raft_consensus.cc:348] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.796506 21360 raft_consensus.cc:374] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:31.796603 21360 raft_consensus.cc:729] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f71377f7ff1c4b68a0fb47673a3eaafe, State: Initialized, Role: FOLLOWER
I20260811 05:12:31.797045 21360 consensus_queue.cc:260] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.797155 21359 ts_tablet_manager.cc:1428] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:12:31.797479 21181 heartbeater.cc:503] Master 127.30.88.254:39765 was elected leader, sending a full tablet report...
I20260811 05:12:31.797904 21360 ts_tablet_manager.cc:1428] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:12:31.798246 21338 heartbeater.cc:503] Master 127.30.88.254:39765 was elected leader, sending a full tablet report...
I20260811 05:12:31.860671 21365 raft_consensus.cc:482] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:31.860913 21365 raft_consensus.cc:504] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.862115 21365 leader_election.cc:288] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f71377f7ff1c4b68a0fb47673a3eaafe (127.30.88.195:33133), 51bb1a3a91ac4cc1b2093a8dad512145 (127.30.88.193:36053)
I20260811 05:12:31.865962 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.862349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:33133 (local address 127.30.88.194:40687)
0811 05:12:31.862513 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.862531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.862633 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.863000 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.863004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.863036 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:31.863164 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.863171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.863989 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.863992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.864824 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.864832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.865489 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.865495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.865618 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.865649 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.865721 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.865818 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"spinlock_wait_cycles":768}
I20260811 05:12:31.865973 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.862620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:40429 (local address 127.30.88.193:36053)
0811 05:12:31.862755 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.862760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.863009 (+ 249us) server_negotiation.cc:407] Connection header received
0811 05:12:31.863108 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.863110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.863165 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.863251 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:31.863530 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.864031 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.865025 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.865454 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.865572 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.865624 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.865802 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.865875 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:12:31.865965 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.862467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:40687 (local address 127.30.88.195:33133)
0811 05:12:31.862609 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.862616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.862636 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:12:31.862741 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.862747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.862800 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.862972 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:31.863284 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.863881 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.864954 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.865363 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.865460 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.865499 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.865769 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.865840 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:12:31.865984 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.862349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36053 (local address 127.30.88.194:40429)
0811 05:12:31.862918 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.862934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:31.863023 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:31.863277 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:31.863280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:31.863296 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:31.863420 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.863426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.864146 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.864148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.864898 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:31.864911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.865582 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.865586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:31.865660 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.865677 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:12:31.865766 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:12:31.865836 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":119,"thread_start_us":156,"threads_started":1}
I20260811 05:12:31.866671 20968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776f1ff2d44c436c83c37fcf1d491ec1" candidate_uuid: "f44f341ce36a4d9590862db401ebe4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" is_pre_election: true
I20260811 05:12:31.866681 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776f1ff2d44c436c83c37fcf1d491ec1" candidate_uuid: "f44f341ce36a4d9590862db401ebe4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" is_pre_election: true
I20260811 05:12:31.867000 20968 raft_consensus.cc:2454] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f44f341ce36a4d9590862db401ebe4cd in term 0.
I20260811 05:12:31.867007 21282 raft_consensus.cc:2454] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f44f341ce36a4d9590862db401ebe4cd in term 0.
I20260811 05:12:31.867443 21052 leader_election.cc:302] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51bb1a3a91ac4cc1b2093a8dad512145, f44f341ce36a4d9590862db401ebe4cd; no voters:
I20260811 05:12:31.867733 21365 raft_consensus.cc:2781] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:31.867851 21365 raft_consensus.cc:482] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:31.867914 21365 raft_consensus.cc:3037] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:31.868963 21365 raft_consensus.cc:504] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.869375 21365 leader_election.cc:288] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [CANDIDATE]: Term 1 election: Requested vote from peers f71377f7ff1c4b68a0fb47673a3eaafe (127.30.88.195:33133), 51bb1a3a91ac4cc1b2093a8dad512145 (127.30.88.193:36053)
I20260811 05:12:31.869675 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776f1ff2d44c436c83c37fcf1d491ec1" candidate_uuid: "f44f341ce36a4d9590862db401ebe4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe"
I20260811 05:12:31.869707 20968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776f1ff2d44c436c83c37fcf1d491ec1" candidate_uuid: "f44f341ce36a4d9590862db401ebe4cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb1a3a91ac4cc1b2093a8dad512145"
I20260811 05:12:31.869799 21282 raft_consensus.cc:3037] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:31.869802 20968 raft_consensus.cc:3037] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:31.871158 20968 raft_consensus.cc:2454] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f44f341ce36a4d9590862db401ebe4cd in term 1.
I20260811 05:12:31.871157 21282 raft_consensus.cc:2454] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f44f341ce36a4d9590862db401ebe4cd in term 1.
I20260811 05:12:31.871471 21048 leader_election.cc:302] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f44f341ce36a4d9590862db401ebe4cd, f71377f7ff1c4b68a0fb47673a3eaafe; no voters:
I20260811 05:12:31.871619 21365 raft_consensus.cc:2781] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:31.871876 21365 raft_consensus.cc:686] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 1 LEADER]: Becoming Leader. State: Replica: f44f341ce36a4d9590862db401ebe4cd, State: Running, Role: LEADER
I20260811 05:12:31.872219 21365 consensus_queue.cc:237] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } }
I20260811 05:12:31.875631 20798 catalog_manager.cc:5495] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd reported cstate change: term changed from 0 to 1, leader changed from <none> to f44f341ce36a4d9590862db401ebe4cd (127.30.88.194). New cstate: current_term: 1 leader_uuid: "f44f341ce36a4d9590862db401ebe4cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f44f341ce36a4d9590862db401ebe4cd" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 33005 } health_report { overall_health: HEALTHY } } }
I20260811 05:12:31.904791 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.901234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.30.88.194:33005)
0811 05:12:31.901364 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:31.901370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:31.901387 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:12:31.901593 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:31.901596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:31.901651 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:31.901763 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:31.901995 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.902473 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.903383 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:31.903601 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:31.903710 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:31.904006 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:31.904397 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:31.904525 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:31.904591 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 05:12:31.904645 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:12:31.910416 20968 raft_consensus.cc:1261] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 1 FOLLOWER]: Refusing update from remote peer f44f341ce36a4d9590862db401ebe4cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:31.910418 21282 raft_consensus.cc:1261] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 1 FOLLOWER]: Refusing update from remote peer f44f341ce36a4d9590862db401ebe4cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:31.911192 21369 consensus_queue.cc:1035] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "51bb1a3a91ac4cc1b2093a8dad512145" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 36053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:31.911386 21365 consensus_queue.cc:1035] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f71377f7ff1c4b68a0fb47673a3eaafe" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 33133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:31.919746 21374 mvcc.cc:205] Tried to move back new op lower bound from 7317197422217822208 to 7317197422071545856. Current Snapshot: MvccSnapshot[applied={T|T < 7317197422217822208}]
I20260811 05:12:31.920919 21375 mvcc.cc:205] Tried to move back new op lower bound from 7317197422217822208 to 7317197422071545856. Current Snapshot: MvccSnapshot[applied={T|T < 7317197422217822208}]
I20260811 05:12:31.928375 20798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:31.928586 20798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:31.931285 20798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99be0a149f314340b6515d63cda848d1: Sending DeleteTablet for 3 replicas of tablet 776f1ff2d44c436c83c37fcf1d491ec1
I20260811 05:12:31.932119 21260 tablet_service.cc:1514] Processing DeleteTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:31 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:60604
I20260811 05:12:31.932165 20941 tablet_service.cc:1514] Processing DeleteTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:31 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:40940
I20260811 05:12:31.932443 21104 tablet_service.cc:1514] Processing DeleteTablet for tablet 776f1ff2d44c436c83c37fcf1d491ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:31 UTC) from {username='kudu', principal='kudu/127.30.88.254@KRBTEST.COM'} at 127.0.0.1:40312
I20260811 05:12:31.932682 21385 tablet_replica.cc:330] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: stopping tablet replica
I20260811 05:12:31.932714 21388 tablet_replica.cc:330] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145: stopping tablet replica
I20260811 05:12:31.933070 21385 raft_consensus.cc:2229] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:31.933089 21388 raft_consensus.cc:2229] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:31.933429 21388 raft_consensus.cc:2258] T 776f1ff2d44c436c83c37fcf1d491ec1 P 51bb1a3a91ac4cc1b2093a8dad512145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:31.933465 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20867
I20260811 05:12:31.933456 21385 raft_consensus.cc:2258] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:31.935547 21385 ts_tablet_manager.cc:1905] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:31.939568 21385 ts_tablet_manager.cc:1918] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:31.939697 21385 log.cc:1196] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/776f1ff2d44c436c83c37fcf1d491ec1
I20260811 05:12:31.940173 21385 ts_tablet_manager.cc:1939] T 776f1ff2d44c436c83c37fcf1d491ec1 P f71377f7ff1c4b68a0fb47673a3eaafe: Deleting consensus metadata
I20260811 05:12:31.942535 21389 tablet_replica.cc:330] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: stopping tablet replica
I20260811 05:12:31.942708 20780 catalog_manager.cc:4841] TS f71377f7ff1c4b68a0fb47673a3eaafe (127.30.88.195:33133): tablet 776f1ff2d44c436c83c37fcf1d491ec1 (table test-table [id=2ba70a2977fa4d56873bc8f1b22c50e7]) successfully deleted
I20260811 05:12:31.942981 21389 raft_consensus.cc:2229] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:31.943434 21389 raft_consensus.cc:2258] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:31.945024 21389 ts_tablet_manager.cc:1905] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:31.947817 21389 ts_tablet_manager.cc:1918] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:31.947964 21389 log.cc:1196] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786424902138073-31075-0/minicluster-data/ts-1/wal/wals/776f1ff2d44c436c83c37fcf1d491ec1
I20260811 05:12:31.948299 21389 ts_tablet_manager.cc:1939] T 776f1ff2d44c436c83c37fcf1d491ec1 P f44f341ce36a4d9590862db401ebe4cd: Deleting consensus metadata
I20260811 05:12:31.949509 20781 catalog_manager.cc:4841] TS f44f341ce36a4d9590862db401ebe4cd (127.30.88.194:33005): tablet 776f1ff2d44c436c83c37fcf1d491ec1 (table test-table [id=2ba70a2977fa4d56873bc8f1b22c50e7]) successfully deleted
W20260811 05:12:31.949855 20783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:36053 (error 108)
I20260811 05:12:31.950927 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:31.950586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:36053 (local address 127.0.0.1:40954)
0811 05:12:31.950729 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:31.950805 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 05:12:31.951052 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21031
W20260811 05:12:31.953032 20783 catalog_manager.cc:4584] TS 51bb1a3a91ac4cc1b2093a8dad512145 (127.30.88.193:36053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 776f1ff2d44c436c83c37fcf1d491ec1: Network error: Client connection negotiation failed: client connection to 127.30.88.193:36053: connect: Connection refused (error 111)
I20260811 05:12:31.959775 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21188
I20260811 05:12:31.969823 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 20762
2026-08-11T05:12:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2438 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T05:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:31Z Disabled control of system clock
I20260811 05:12:32.004602 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35717
--webserver_interface=127.30.88.254
--webserver_port=0
--builtin_ntp_servers=127.30.88.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:32.119923 21395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:32.120209 21395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:32.120260 21395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:32.124369 21395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:32.124452 21395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:32.124471 21395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:32.124536 21395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:32.124570 21395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:32.130489 21395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:32.131723 21395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:32.133194 21395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:32.141956 21401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.141956 21404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.141991 21400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:32.142314 21395 server_base.cc:1034] running on GCE node
I20260811 05:12:32.143023 21395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:32.144158 21395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:32.145344 21395 hybrid_clock.cc:648] HybridClock initialized: now 1786425152145320 us; error 46 us; skew 500 ppm
I20260811 05:12:32.147197 21395 webserver.cc:460] Webserver started at http://127.30.88.254:32931/ using document root <none> and password file <none>
I20260811 05:12:32.147782 21395 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:32.147879 21395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:32.148114 21395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:32.150187 21395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "10fc0d8fad7046f0891c851d03d51968"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "6f45c144a1aae501760c05d222f4dfb9"
server_key_iv: "ab6baf218cee0fd0e52c531747eaa9af"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.150787 21395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "10fc0d8fad7046f0891c851d03d51968"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "6f45c144a1aae501760c05d222f4dfb9"
server_key_iv: "ab6baf218cee0fd0e52c531747eaa9af"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.154441 21395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:32.157007 21410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.158293 21395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:32.158481 21395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "10fc0d8fad7046f0891c851d03d51968"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "6f45c144a1aae501760c05d222f4dfb9"
server_key_iv: "ab6baf218cee0fd0e52c531747eaa9af"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.158609 21395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:32.193652 21395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:32.194444 21395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:32.194672 21395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:32.202227 21395 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35717
I20260811 05:12:32.202240 21469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35717 every 8 connection(s)
I20260811 05:12:32.203276 21395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:32.206430 21471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:32.211329 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21395
I20260811 05:12:32.211576 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:32.211882 31075 external_mini_cluster.cc:1428] Setting key 456feb6e8b80cf2b5c262ff808def593
I20260811 05:12:32.212399 21471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: Bootstrap starting.
I20260811 05:12:32.214807 21471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:32.215664 21471 log.cc:824] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:32.217911 21471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: No bootstrap required, opened a new log
I20260811 05:12:32.221650 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.213464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.30.88.254:35717)
0811 05:12:32.214122 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.214136 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.214169 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:12:32.214833 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.214924 (+ 91us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.215343 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.215632 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:32.216050 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.217011 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.218236 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.218754 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.218985 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.219828 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:32.219833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:32.219847 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:32.220249 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:32.220306 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:32.220326 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:32.220463 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.220606 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.220878 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":167,"threads_started":1}
I20260811 05:12:32.222240 21471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } }
I20260811 05:12:32.222649 21471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:32.222841 21471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc0d8fad7046f0891c851d03d51968, State: Initialized, Role: FOLLOWER
I20260811 05:12:32.223332 21471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } }
I20260811 05:12:32.223531 21471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:32.223614 21471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:32.223742 21471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:32.224905 21471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } }
I20260811 05:12:32.225361 21471 leader_election.cc:302] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10fc0d8fad7046f0891c851d03d51968; no voters:
I20260811 05:12:32.225798 21471 leader_election.cc:288] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:32.225880 21476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:32.226094 21476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [term 1 LEADER]: Becoming Leader. State: Replica: 10fc0d8fad7046f0891c851d03d51968, State: Running, Role: LEADER
I20260811 05:12:32.226480 21476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } }
I20260811 05:12:32.227267 21471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:32.228367 21479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fc0d8fad7046f0891c851d03d51968. Latest consensus state: current_term: 1 leader_uuid: "10fc0d8fad7046f0891c851d03d51968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } } }
I20260811 05:12:32.228516 21479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:32.228811 21477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10fc0d8fad7046f0891c851d03d51968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0d8fad7046f0891c851d03d51968" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35717 } } }
I20260811 05:12:32.228924 21477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:32.228929 21486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:32.232321 21486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:32.237946 21486 catalog_manager.cc:1347] Generated new cluster ID: 4b38b1f391f3476399106a3a52a625ac
I20260811 05:12:32.238029 21486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:32.248733 21486 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:32.250022 21486 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:32.255481 21486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: Generated new TSK 0
I20260811 05:12:32.256302 21486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:32.262910 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.193:0
--local_ip_for_outbound_sockets=127.30.88.193
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--builtin_ntp_servers=127.30.88.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:12:32.377285 21497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:32.377592 21497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:32.377671 21497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:32.381740 21497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:32.381826 21497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:32.381919 21497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.193
I20260811 05:12:32.386636 21497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.88.193
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.193
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:32.387786 21497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:32.389178 21497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:32.399906 21505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.399919 21503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:32.400002 21497 server_base.cc:1034] running on GCE node
W20260811 05:12:32.399922 21507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:32.400718 21497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:32.401363 21497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:32.402573 21497 hybrid_clock.cc:648] HybridClock initialized: now 1786425152402552 us; error 32 us; skew 500 ppm
I20260811 05:12:32.404572 21497 webserver.cc:460] Webserver started at http://127.30.88.193:37479/ using document root <none> and password file <none>
I20260811 05:12:32.405187 21497 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:32.405253 21497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:32.405474 21497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:32.407616 21497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data/instance:
uuid: "3415f8edfa6b4f50ae6683a786def77f"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "28390537347946249ebd55f8d9e12ea9"
server_key_iv: "20491c38c3bff931640b554cb6a356f3"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.408201 21497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance:
uuid: "3415f8edfa6b4f50ae6683a786def77f"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "28390537347946249ebd55f8d9e12ea9"
server_key_iv: "20491c38c3bff931640b554cb6a356f3"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.411928 21497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:12:32.414259 21515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.415483 21497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:32.415621 21497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal
uuid: "3415f8edfa6b4f50ae6683a786def77f"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "28390537347946249ebd55f8d9e12ea9"
server_key_iv: "20491c38c3bff931640b554cb6a356f3"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.415727 21497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:32.439853 21497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:32.440543 21497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:32.440738 21497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:32.441321 21497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:32.442368 21497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:32.442451 21497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.442524 21497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:32.442567 21497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.452348 21497 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.193:37773
I20260811 05:12:32.452375 21649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.193:37773 every 8 connection(s)
I20260811 05:12:32.453248 21497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:32.460542 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21497
I20260811 05:12:32.460668 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:32.461009 31075 external_mini_cluster.cc:1428] Setting key 02132f1d1e536c0eb4977fd2f3cb0483
I20260811 05:12:32.463058 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.194:0
--local_ip_for_outbound_sockets=127.30.88.194
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--builtin_ntp_servers=127.30.88.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:32.467519 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.455151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.193:46943 (local address 127.30.88.254:35717)
0811 05:12:32.455297 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.455307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.456269 (+ 962us) server_negotiation.cc:407] Connection header received
0811 05:12:32.457619 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.457621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.457669 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.457745 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:32.458693 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.459324 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.460422 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.460756 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.460873 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.467028 (+ 6155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:32.467034 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:32.467040 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:32.467053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:32.467072 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:32.467079 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:32.467138 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.467356 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.467414 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:12:32.468775 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.455482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35717 (local address 127.30.88.193:46943)
0811 05:12:32.456117 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.456158 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.457425 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.457970 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.457977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.458069 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:32.458556 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.458569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.459435 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.459437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.460299 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.460308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.460910 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.460912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.461100 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.466687 (+ 5587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:32.466691 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:32.466744 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:32.466843 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:32.467169 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:32.467181 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:32.467188 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.467351 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.467664 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260811 05:12:32.470151 21650 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35717
I20260811 05:12:32.470413 21650 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:32.471006 21650 heartbeater.cc:511] Master 127.30.88.254:35717 requested a full tablet report, sending...
I20260811 05:12:32.472488 21430 ts_manager.cc:194] Registered new tserver with Master: 3415f8edfa6b4f50ae6683a786def77f (127.30.88.193:37773)
I20260811 05:12:32.473598 21430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.193:46943
W20260811 05:12:32.573736 21654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:32.574028 21654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:32.574098 21654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:32.577983 21654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:32.578073 21654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:32.578168 21654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.194
I20260811 05:12:32.582903 21654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.88.194
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.194
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:32.584007 21654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:32.585315 21654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:32.594642 21661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.594697 21660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.594642 21663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:32.595371 21654 server_base.cc:1034] running on GCE node
I20260811 05:12:32.595767 21654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:32.596364 21654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:32.597609 21654 hybrid_clock.cc:648] HybridClock initialized: now 1786425152597583 us; error 31 us; skew 500 ppm
I20260811 05:12:32.599545 21654 webserver.cc:460] Webserver started at http://127.30.88.194:43691/ using document root <none> and password file <none>
I20260811 05:12:32.600126 21654 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:32.600219 21654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:32.600461 21654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:32.602487 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data/instance:
uuid: "2420857c3cb34c53a4bcea338e40eafa"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "a2a152643c3c20dab6f6e03a30fcce12"
server_key_iv: "da803925cebe6a756071891637cacd10"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.603101 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance:
uuid: "2420857c3cb34c53a4bcea338e40eafa"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "a2a152643c3c20dab6f6e03a30fcce12"
server_key_iv: "da803925cebe6a756071891637cacd10"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.606678 21654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:12:32.609035 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.610234 21654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:32.610402 21654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal
uuid: "2420857c3cb34c53a4bcea338e40eafa"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "a2a152643c3c20dab6f6e03a30fcce12"
server_key_iv: "da803925cebe6a756071891637cacd10"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.610519 21654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:32.625968 21654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:32.626739 21654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:32.627004 21654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:32.627614 21654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:32.628747 21654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:32.628796 21654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.628862 21654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:32.628911 21654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.637424 21654 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.194:34381
I20260811 05:12:32.637447 21803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.194:34381 every 8 connection(s)
I20260811 05:12:32.638314 21654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:32.639808 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21654
I20260811 05:12:32.639957 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:32.640309 31075 external_mini_cluster.cc:1428] Setting key 888b784e16160af09cdcca101ad6e438
I20260811 05:12:32.642445 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.88.195:0
--local_ip_for_outbound_sockets=127.30.88.195
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--builtin_ntp_servers=127.30.88.212:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:12:32.648089 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.640210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:50709 (local address 127.30.88.254:35717)
0811 05:12:32.641656 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.641663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.641681 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:12:32.643319 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.643325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.643391 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.643487 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:32.644332 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.644860 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.646033 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.646255 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.646375 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.647559 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:32.647562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:32.647565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:32.647584 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:32.647603 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:32.647611 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:32.647714 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.647876 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.647937 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334}
I20260811 05:12:32.648890 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.640718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35717 (local address 127.30.88.194:50709)
0811 05:12:32.641520 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.641586 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.643101 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.643644 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.643650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.643729 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:32.644192 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.644204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.644985 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.644988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.645906 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.645916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.646425 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.646430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.646604 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.647300 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:32.647302 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:32.647339 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:32.647398 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:32.647716 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:32.647728 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:32.647737 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.647875 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.648140 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":170,"threads_started":1}
I20260811 05:12:32.650015 21804 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35717
I20260811 05:12:32.650305 21804 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:32.650794 21804 heartbeater.cc:511] Master 127.30.88.254:35717 requested a full tablet report, sending...
I20260811 05:12:32.651886 21430 ts_manager.cc:194] Registered new tserver with Master: 2420857c3cb34c53a4bcea338e40eafa (127.30.88.194:34381)
I20260811 05:12:32.652769 21430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.194:50709
W20260811 05:12:32.753319 21807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:32.753530 21807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:32.753564 21807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:32.757486 21807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:32.757534 21807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:32.757584 21807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.88.195
I20260811 05:12:32.762238 21807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.88.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.88.195
--webserver_port=0
--tserver_master_addrs=127.30.88.254:35717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.88.195
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:32.763271 21807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:32.764535 21807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:32.773521 21815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.773540 21813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:32.773582 21812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:32.773802 21807 server_base.cc:1034] running on GCE node
I20260811 05:12:32.774216 21807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:32.774775 21807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:32.775992 21807 hybrid_clock.cc:648] HybridClock initialized: now 1786425152775972 us; error 30 us; skew 500 ppm
I20260811 05:12:32.777791 21807 webserver.cc:460] Webserver started at http://127.30.88.195:37359/ using document root <none> and password file <none>
I20260811 05:12:32.778342 21807 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:32.778429 21807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:32.778656 21807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:32.780637 21807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data/instance:
uuid: "1ec9998f54ac4b6094443e67dbbb31ab"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "478dc345071cda3c38abfd8c3f4966b7"
server_key_iv: "e2b95def9ad952570508d98567a91cd8"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.781189 21807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance:
uuid: "1ec9998f54ac4b6094443e67dbbb31ab"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "478dc345071cda3c38abfd8c3f4966b7"
server_key_iv: "e2b95def9ad952570508d98567a91cd8"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.784704 21807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:32.786967 21822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.788204 21807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:32.788362 21807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal
uuid: "1ec9998f54ac4b6094443e67dbbb31ab"
format_stamp: "Formatted at 2026-08-11 05:12:32 on dist-test-slave-7c35"
server_key: "478dc345071cda3c38abfd8c3f4966b7"
server_key_iv: "e2b95def9ad952570508d98567a91cd8"
server_key_version: "encryptionkey@0"
I20260811 05:12:32.788482 21807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:32.805817 21807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:32.806416 21807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:32.806593 21807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:32.807219 21807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:32.808158 21807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:32.808229 21807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.808293 21807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:32.808341 21807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:32.817871 21807 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.195:37099
I20260811 05:12:32.817894 21958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.195:37099 every 8 connection(s)
I20260811 05:12:32.818758 21807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:32.819101 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 21807
I20260811 05:12:32.819248 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:32.819633 31075 external_mini_cluster.cc:1428] Setting key 6da7e96f2d36f0161281d7a615634c9d
I20260811 05:12:32.828325 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.820723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.195:52529 (local address 127.30.88.254:35717)
0811 05:12:32.820869 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.820881 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.822187 (+ 1306us) server_negotiation.cc:407] Connection header received
0811 05:12:32.823631 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.823634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.823692 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.823765 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:32.824608 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.825138 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.826255 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.826492 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.826642 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.827803 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:32.827807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:32.827809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:32.827822 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:32.827849 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:32.827857 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:32.827937 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.828120 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.828180 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:12:32.829223 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.821253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:35717 (local address 127.30.88.195:52529)
0811 05:12:32.822029 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.822092 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.823426 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.823933 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.823940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.824019 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:32.824478 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.824489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.825264 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.825267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.826115 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.826124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.826682 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.826686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.826921 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.827506 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:12:32.827507 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:32.827547 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:12:32.827661 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:32.827940 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:32.827951 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:32.827958 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.828104 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.828396 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":125,"threads_started":1}
I20260811 05:12:32.830333 21959 heartbeater.cc:347] Connected to a master server at 127.30.88.254:35717
I20260811 05:12:32.830595 21959 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:32.831180 21959 heartbeater.cc:511] Master 127.30.88.254:35717 requested a full tablet report, sending...
I20260811 05:12:32.832244 21430 ts_manager.cc:194] Registered new tserver with Master: 1ec9998f54ac4b6094443e67dbbb31ab (127.30.88.195:37099)
I20260811 05:12:32.832823 21430 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.88.195:52529
I20260811 05:12:32.834422 31075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:32.840205 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.836420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.30.88.254:35717)
0811 05:12:32.836576 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.836580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.836755 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:12:32.836903 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.836905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.836943 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.837022 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:32.837304 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.837814 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.838660 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.838937 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.839039 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.839788 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:32.839792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:32.839795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:32.839807 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:32.839825 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:32.839840 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:32.839917 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.840010 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.840060 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:12:32.843237 21430 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:32.846128 21430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:32.860357 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.854317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34381 (local address 127.0.0.1:35512)
0811 05:12:32.855096 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.855146 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.855288 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.855957 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.855963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.855982 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:32.856131 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.856138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857854 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.857857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.858979 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.858989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.859808 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.859811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.859922 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.859961 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.860075 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.860184 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":76,"thread_start_us":66,"threads_started":1}
I20260811 05:12:32.860365 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.854457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.30.88.194:34381)
0811 05:12:32.854780 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.854792 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.855442 (+ 650us) server_negotiation.cc:407] Connection header received
0811 05:12:32.855550 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.855558 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.855779 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.855932 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:32.856276 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857479 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.859125 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.859692 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.859795 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.859977 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.860095 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.860184 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":113,"threads_started":1}
I20260811 05:12:32.861255 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.854403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:37773 (local address 127.0.0.1:59432)
0811 05:12:32.855709 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.855719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.855829 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.856256 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.856259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.856277 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:32.856364 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.856369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857693 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.857697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.859497 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.859508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.860731 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.860734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.860846 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.860869 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.860941 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.861017 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":179,"thread_start_us":112,"threads_started":1}
I20260811 05:12:32.861748 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.855144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.30.88.193:37773)
0811 05:12:32.855598 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.855612 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.855806 (+ 194us) server_negotiation.cc:407] Connection header received
0811 05:12:32.855937 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.855942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.856113 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.856262 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:32.856496 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857589 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.859631 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.860402 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.860508 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.860676 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.861494 (+ 818us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.861592 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":110,"threads_started":1}
I20260811 05:12:32.861917 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.855429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.30.88.195:37099)
0811 05:12:32.856166 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:32.856174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:32.856213 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:12:32.856282 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:32.856287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:32.856457 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:32.856629 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:32.856850 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857618 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.859638 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.860415 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.860879 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.861067 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:32.861659 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 05:12:32.861748 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":117,"threads_started":1}
I20260811 05:12:32.862154 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:32.855039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:37099 (local address 127.0.0.1:45834)
0811 05:12:32.855832 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:32.855847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:32.855924 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:32.856619 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:32.856621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:32.856639 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:32.856730 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.856739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.857744 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.857746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.859497 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:32.859508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:32.861147 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:32.861151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:32.861262 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:32.861285 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:12:32.861633 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 05:12:32.861705 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":254,"thread_start_us":79,"threads_started":1}
I20260811 05:12:32.863811 21728 tablet_service.cc:1467] Processing CreateTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 (DEFAULT_TABLE table=test-table [id=02dfdad3ea3d45b8a44583a17d99ac5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:32.864455 21571 tablet_service.cc:1467] Processing CreateTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 (DEFAULT_TABLE table=test-table [id=02dfdad3ea3d45b8a44583a17d99ac5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:32.865085 21728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8093b124a5247aeaaee25b3a657c9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:32.865460 21571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8093b124a5247aeaaee25b3a657c9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:32.865254 21880 tablet_service.cc:1467] Processing CreateTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 (DEFAULT_TABLE table=test-table [id=02dfdad3ea3d45b8a44583a17d99ac5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:32.866106 21880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8093b124a5247aeaaee25b3a657c9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:32.871881 21978 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Bootstrap starting.
I20260811 05:12:32.873116 21979 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Bootstrap starting.
I20260811 05:12:32.873978 21978 tablet_bootstrap.cc:654] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:32.874655 21978 log.cc:824] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:32.874771 21979 tablet_bootstrap.cc:654] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:32.875128 21980 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Bootstrap starting.
I20260811 05:12:32.875586 21979 log.cc:824] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:32.876648 21978 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: No bootstrap required, opened a new log
I20260811 05:12:32.876825 21978 ts_tablet_manager.cc:1397] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:12:32.876979 21980 tablet_bootstrap.cc:654] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:32.877312 21979 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: No bootstrap required, opened a new log
I20260811 05:12:32.877504 21979 ts_tablet_manager.cc:1397] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:12:32.877753 21980 log.cc:824] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:32.879662 21980 tablet_bootstrap.cc:492] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: No bootstrap required, opened a new log
I20260811 05:12:32.879824 21980 ts_tablet_manager.cc:1397] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 05:12:32.880748 21979 raft_consensus.cc:348] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.880971 21979 raft_consensus.cc:374] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:32.881022 21979 raft_consensus.cc:729] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2420857c3cb34c53a4bcea338e40eafa, State: Initialized, Role: FOLLOWER
I20260811 05:12:32.880904 21978 raft_consensus.cc:348] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.881131 21978 raft_consensus.cc:374] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:32.881187 21978 raft_consensus.cc:729] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec9998f54ac4b6094443e67dbbb31ab, State: Initialized, Role: FOLLOWER
I20260811 05:12:32.881507 21979 consensus_queue.cc:260] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.881580 21978 consensus_queue.cc:260] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.882469 21978 ts_tablet_manager.cc:1428] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:12:32.882483 21979 ts_tablet_manager.cc:1428] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:12:32.882818 21959 heartbeater.cc:503] Master 127.30.88.254:35717 was elected leader, sending a full tablet report...
I20260811 05:12:32.882848 21804 heartbeater.cc:503] Master 127.30.88.254:35717 was elected leader, sending a full tablet report...
I20260811 05:12:32.883513 21980 raft_consensus.cc:348] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.883677 21980 raft_consensus.cc:374] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:32.883713 21980 raft_consensus.cc:729] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3415f8edfa6b4f50ae6683a786def77f, State: Initialized, Role: FOLLOWER
I20260811 05:12:32.884172 21980 consensus_queue.cc:260] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:32.884809 21650 heartbeater.cc:503] Master 127.30.88.254:35717 was elected leader, sending a full tablet report...
I20260811 05:12:32.885159 21980 ts_tablet_manager.cc:1428] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 05:12:32.889240 21805 tablet.cc:2369] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:32.954509 21652 tablet.cc:2369] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:33.027271 21984 raft_consensus.cc:482] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:33.027493 21984 raft_consensus.cc:504] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:33.028836 21984 leader_election.cc:288] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3415f8edfa6b4f50ae6683a786def77f (127.30.88.193:37773), 1ec9998f54ac4b6094443e67dbbb31ab (127.30.88.195:37099)
I20260811 05:12:33.032753 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.029193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:57093 (local address 127.30.88.193:37773)
0811 05:12:33.029381 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:33.029388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:33.029413 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:12:33.029486 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:33.029490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:33.029559 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:33.029670 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:33.029985 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.030556 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.031741 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.032153 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.032299 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:33.032373 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:33.032551 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:12:33.032620 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:12:33.032754 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.029067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:37773 (local address 127.30.88.194:57093)
0811 05:12:33.029278 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:33.029295 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:33.029392 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:33.029684 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:33.029687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:33.029707 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:33.029851 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:33.029858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.030688 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.030694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:33.031596 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:33.031606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.032287 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.032291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:33.032405 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:33.032431 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:12:33.032534 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:12:33.032609 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":6}
I20260811 05:12:33.032790 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.029450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.88.194:42975 (local address 127.30.88.195:37099)
0811 05:12:33.029585 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:33.029590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:33.029736 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:12:33.029890 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:33.029892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:33.029931 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:33.030012 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:33.030320 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.030823 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.031890 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.032272 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.032396 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:33.032440 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:33.032614 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 05:12:33.032682 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:12:33.032804 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.029206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.195:37099 (local address 127.30.88.194:42975)
0811 05:12:33.029648 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:33.029658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:33.029771 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:33.030024 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:33.030027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:33.030049 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:33.030208 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:33.030213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.030974 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.030979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:33.031769 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:33.031778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.032390 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.032393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:33.032485 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:33.032505 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:12:33.032598 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:12:33.032660 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":38,"thread_start_us":170,"threads_started":1}
I20260811 05:12:33.033524 21594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8093b124a5247aeaaee25b3a657c9b2" candidate_uuid: "2420857c3cb34c53a4bcea338e40eafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3415f8edfa6b4f50ae6683a786def77f" is_pre_election: true
I20260811 05:12:33.033538 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8093b124a5247aeaaee25b3a657c9b2" candidate_uuid: "2420857c3cb34c53a4bcea338e40eafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" is_pre_election: true
I20260811 05:12:33.033821 21594 raft_consensus.cc:2454] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2420857c3cb34c53a4bcea338e40eafa in term 0.
I20260811 05:12:33.033821 21902 raft_consensus.cc:2454] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2420857c3cb34c53a4bcea338e40eafa in term 0.
I20260811 05:12:33.034324 21670 leader_election.cc:302] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1ec9998f54ac4b6094443e67dbbb31ab, 2420857c3cb34c53a4bcea338e40eafa, 3415f8edfa6b4f50ae6683a786def77f; no voters:
I20260811 05:12:33.034601 21984 raft_consensus.cc:2781] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:33.034683 21984 raft_consensus.cc:482] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:33.034710 21984 raft_consensus.cc:3037] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:33.035832 21984 raft_consensus.cc:504] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:33.036216 21984 leader_election.cc:288] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [CANDIDATE]: Term 1 election: Requested vote from peers 3415f8edfa6b4f50ae6683a786def77f (127.30.88.193:37773), 1ec9998f54ac4b6094443e67dbbb31ab (127.30.88.195:37099)
I20260811 05:12:33.036610 21594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8093b124a5247aeaaee25b3a657c9b2" candidate_uuid: "2420857c3cb34c53a4bcea338e40eafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3415f8edfa6b4f50ae6683a786def77f"
I20260811 05:12:33.036650 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8093b124a5247aeaaee25b3a657c9b2" candidate_uuid: "2420857c3cb34c53a4bcea338e40eafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec9998f54ac4b6094443e67dbbb31ab"
I20260811 05:12:33.036727 21594 raft_consensus.cc:3037] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:33.036816 21902 raft_consensus.cc:3037] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:33.037725 21594 raft_consensus.cc:2454] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2420857c3cb34c53a4bcea338e40eafa in term 1.
I20260811 05:12:33.037941 21902 raft_consensus.cc:2454] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2420857c3cb34c53a4bcea338e40eafa in term 1.
I20260811 05:12:33.038082 21672 leader_election.cc:302] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2420857c3cb34c53a4bcea338e40eafa, 3415f8edfa6b4f50ae6683a786def77f; no voters:
I20260811 05:12:33.038256 21984 raft_consensus.cc:2781] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:33.038511 21984 raft_consensus.cc:686] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [term 1 LEADER]: Becoming Leader. State: Replica: 2420857c3cb34c53a4bcea338e40eafa, State: Running, Role: LEADER
I20260811 05:12:33.038887 21984 consensus_queue.cc:237] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } }
I20260811 05:12:33.041940 21429 catalog_manager.cc:5495] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2420857c3cb34c53a4bcea338e40eafa (127.30.88.194). New cstate: current_term: 1 leader_uuid: "2420857c3cb34c53a4bcea338e40eafa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2420857c3cb34c53a4bcea338e40eafa" member_type: VOTER last_known_addr { host: "127.30.88.194" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 } health_report { overall_health: UNKNOWN } } }
W20260811 05:12:33.070032 21960 tablet.cc:2369] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:33.094035 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.090421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.30.88.194:34381)
0811 05:12:33.090566 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:33.090570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:33.090586 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:12:33.090734 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:33.090737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:33.090793 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:33.090910 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:33.091148 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.091669 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.092524 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:33.092742 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:33.092824 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:33.093138 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:33.093642 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:33.093737 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:33.093814 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 05:12:33.093860 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:12:33.100021 21594 raft_consensus.cc:1261] T d8093b124a5247aeaaee25b3a657c9b2 P 3415f8edfa6b4f50ae6683a786def77f [term 1 FOLLOWER]: Refusing update from remote peer 2420857c3cb34c53a4bcea338e40eafa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:33.100020 21902 raft_consensus.cc:1261] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 1 FOLLOWER]: Refusing update from remote peer 2420857c3cb34c53a4bcea338e40eafa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:33.100823 21989 consensus_queue.cc:1035] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3415f8edfa6b4f50ae6683a786def77f" member_type: VOTER last_known_addr { host: "127.30.88.193" port: 37773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:33.101006 21984 consensus_queue.cc:1035] T d8093b124a5247aeaaee25b3a657c9b2 P 2420857c3cb34c53a4bcea338e40eafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ec9998f54ac4b6094443e67dbbb31ab" member_type: VOTER last_known_addr { host: "127.30.88.195" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:33.108734 21993 mvcc.cc:205] Tried to move back new op lower bound from 7317197427089973248 to 7317197426850095104. Current Snapshot: MvccSnapshot[applied={T|T < 7317197427089973248}]
I20260811 05:12:33.110685 21994 mvcc.cc:205] Tried to move back new op lower bound from 7317197427089973248 to 7317197426850095104. Current Snapshot: MvccSnapshot[applied={T|T < 7317197427089973248}]
I20260811 05:12:33.118216 21430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:33.118358 21430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:33.120889 21430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10fc0d8fad7046f0891c851d03d51968: Sending DeleteTablet for 3 replicas of tablet d8093b124a5247aeaaee25b3a657c9b2
I20260811 05:12:33.121631 21571 tablet_service.cc:1514] Processing DeleteTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:33 UTC) from {username='slave'} at 127.0.0.1:59432
I20260811 05:12:33.121688 21728 tablet_service.cc:1514] Processing DeleteTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:33 UTC) from {username='slave'} at 127.0.0.1:35512
I20260811 05:12:33.121714 21880 tablet_service.cc:1514] Processing DeleteTablet for tablet d8093b124a5247aeaaee25b3a657c9b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:33 UTC) from {username='slave'} at 127.0.0.1:45834
I20260811 05:12:33.122131 22003 tablet_replica.cc:330] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: stopping tablet replica
I20260811 05:12:33.122459 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21497
I20260811 05:12:33.122474 22003 raft_consensus.cc:2229] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:33.124498 22003 raft_consensus.cc:2258] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:33.127316 22003 ts_tablet_manager.cc:1905] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:33.130338 22003 ts_tablet_manager.cc:1918] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:33.130443 22003 log.cc:1196] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Deleting WAL directory at /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786424902138073-31075-0/minicluster-data/ts-2/wal/wals/d8093b124a5247aeaaee25b3a657c9b2
I20260811 05:12:33.130760 22003 ts_tablet_manager.cc:1939] T d8093b124a5247aeaaee25b3a657c9b2 P 1ec9998f54ac4b6094443e67dbbb31ab: Deleting consensus metadata
I20260811 05:12:33.132061 21411 catalog_manager.cc:4841] TS 1ec9998f54ac4b6094443e67dbbb31ab (127.30.88.195:37099): tablet d8093b124a5247aeaaee25b3a657c9b2 (table test-table [id=02dfdad3ea3d45b8a44583a17d99ac5e]) successfully deleted
W20260811 05:12:33.133795 21412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.88.193:37773 (error 108)
I20260811 05:12:33.134706 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21654
I20260811 05:12:33.137939 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.134589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.193:37773 (local address 127.0.0.1:59440)
0811 05:12:33.134730 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:33.134809 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.193:37773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 05:12:33.138116 21412 catalog_manager.cc:4584] TS 3415f8edfa6b4f50ae6683a786def77f (127.30.88.193:37773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8093b124a5247aeaaee25b3a657c9b2: Network error: Client connection negotiation failed: client connection to 127.30.88.193:37773: connect: Connection refused (error 111)
I20260811 05:12:33.142992 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21807
I20260811 05:12:33.145519 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:33.143177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.194:34381 (local address 127.0.0.1:35532)
0811 05:12:33.145348 (+ 2171us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:33.145419 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.194:34381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2049}
I20260811 05:12:33.149940 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 21395
2026-08-11T05:12:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1178 ms)
[----------] 6 tests from EncryptionPolicyTest (10819 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:33 dist-test-slave-7c35 krb5kdc[22009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:33 dist-test-slave-7c35 krb5kdc[22009](info): set up 2 sockets
Aug 11 05:12:33 dist-test-slave-7c35 krb5kdc[22009](info): commencing operation
krb5kdc: starting...
W20260811 05:12:34.269160 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:34.394289 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:34.502653 22025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:34.502978 22025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:34.503067 22025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:34.506650 22025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:34.506726 22025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:34.506762 22025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:34.506793 22025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:34.506824 22025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:34.511961 22025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:34.512941 22025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:34.513913 22025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:34.522166 22035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:34.522197 22033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:34.522166 22031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:34.522449 22025 server_base.cc:1034] running on GCE node
I20260811 05:12:34.523053 22025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:34.524113 22025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:34.525303 22025 hybrid_clock.cc:648] HybridClock initialized: now 1786425154525278 us; error 35 us; skew 500 ppm
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425154, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:34.528013 22025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:34.528888 22025 webserver.cc:460] Webserver started at http://127.0.0.1:43933/ using document root <none> and password file <none>
I20260811 05:12:34.529454 22025 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:34.529547 22025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:34.529772 22025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:34.531899 22025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "5a6849dcf27140538db60e7ec59259e1"
format_stamp: "Formatted at 2026-08-11 05:12:34 on dist-test-slave-7c35"
server_key: "9d5792151b20b45cf34e16633e41c0c4"
server_key_iv: "4d50ae9969721642acebaa065946e467"
server_key_version: "encryptionkey@0"
I20260811 05:12:34.532469 22025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "5a6849dcf27140538db60e7ec59259e1"
format_stamp: "Formatted at 2026-08-11 05:12:34 on dist-test-slave-7c35"
server_key: "9d5792151b20b45cf34e16633e41c0c4"
server_key_iv: "4d50ae9969721642acebaa065946e467"
server_key_version: "encryptionkey@0"
I20260811 05:12:34.536022 22025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:12:34.538345 22043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:34.539584 22025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:34.539721 22025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "5a6849dcf27140538db60e7ec59259e1"
format_stamp: "Formatted at 2026-08-11 05:12:34 on dist-test-slave-7c35"
server_key: "9d5792151b20b45cf34e16633e41c0c4"
server_key_iv: "4d50ae9969721642acebaa065946e467"
server_key_version: "encryptionkey@0"
I20260811 05:12:34.539824 22025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:34.555106 22025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:34.558048 22025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:34.558269 22025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:34.564922 22025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37607
I20260811 05:12:34.564977 22100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37607 every 8 connection(s)
I20260811 05:12:34.565894 22025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:34.568570 22101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:34.570699 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22025
I20260811 05:12:34.570865 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:34.571178 31075 external_mini_cluster.cc:1428] Setting key b77db83f310a9e76d9643c49146beaee
I20260811 05:12:34.574693 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1: Bootstrap starting.
I20260811 05:12:34.576820 22101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:34.577530 22101 log.cc:824] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:34.579345 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1: No bootstrap required, opened a new log
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:12:34.582882 22101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } }
I20260811 05:12:34.583107 22101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:34.583164 22101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6849dcf27140538db60e7ec59259e1, State: Initialized, Role: FOLLOWER
I20260811 05:12:34.583626 22101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } }
I20260811 05:12:34.583739 22101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:34.583811 22101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:34.583921 22101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:34.584782 22101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } }
I20260811 05:12:34.585165 22101 leader_election.cc:302] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a6849dcf27140538db60e7ec59259e1; no voters:
I20260811 05:12:34.585516 22101 leader_election.cc:288] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:34.585893 22109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:34.586176 22109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5a6849dcf27140538db60e7ec59259e1, State: Running, Role: LEADER
I20260811 05:12:34.586467 22109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } }
I20260811 05:12:34.586711 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:34.572583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44155 (local address 127.0.0.1:37607)
0811 05:12:34.573011 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:34.573025 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:34.573061 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:12:34.573804 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:34.573823 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:34.574245 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:34.574572 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:34.575428 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:34.576377 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:34.577338 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:34.577843 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:34.578014 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:34.580879 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:34.580905 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:34.580918 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:34.580945 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:34.583384 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:34.583898 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:34.583904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:34.583909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:34.584085 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:34.584468 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:34.584471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:34.584474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:34.584855 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:34.585090 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:34.585409 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 05:12:34.585657 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":127,"threads_started":1}
I20260811 05:12:34.586828 22101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:34.588992 22109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a6849dcf27140538db60e7ec59259e1. Latest consensus state: current_term: 1 leader_uuid: "5a6849dcf27140538db60e7ec59259e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } } }
I20260811 05:12:34.589458 22108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a6849dcf27140538db60e7ec59259e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a6849dcf27140538db60e7ec59259e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37607 } } }
I20260811 05:12:34.589771 22108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:34.589990 22109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:34.590189 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:34.590299 22125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:34.593271 22125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:34.599025 22125 catalog_manager.cc:1347] Generated new cluster ID: b52f340fb56e4e1f938fb6fd87d8a3e3
I20260811 05:12:34.599089 22125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:34.605063 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:34.595333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35181 (local address 127.0.0.1:37607)
0811 05:12:34.595461 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:34.595466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:34.595762 (+ 296us) server_negotiation.cc:407] Connection header received
0811 05:12:34.595925 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:34.595928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:34.595978 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:34.596063 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:34.596788 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:34.597875 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:34.599183 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:34.599557 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:34.599679 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:34.601311 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:34.601315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:34.601318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:34.601356 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:34.603190 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:34.603701 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:34.603705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:34.603707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:34.603803 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:34.604149 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:34.604153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:34.604155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:34.604385 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:34.604566 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:34.604806 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 05:12:34.604906 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:12:34.610262 22125 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:34.611155 22125 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:34.619053 22125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a6849dcf27140538db60e7ec59259e1: Generated new TSK 0
I20260811 05:12:34.619783 22125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:34.630642 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22025
W20260811 05:12:34.640008 22006 connection.cc:535] client connection to 127.0.0.1:37607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37607): Connection reset by peer (error 104)
2026-08-11T05:12:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22137](info): set up 2 sockets
Aug 11 05:12:34 dist-test-slave-7c35 krb5kdc[22137](info): commencing operation
krb5kdc: starting...
W20260811 05:12:35.728421 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:35 dist-test-slave-7c35 krb5kdc[22137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:35.857535 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:35.970983 22151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:35.971262 22151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:35.971307 22151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:35.975320 22151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:35.975385 22151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:35.975404 22151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:35.975420 22151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:35.975436 22151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:35.980736 22151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:35.981808 22151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:35.982833 22151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:35.991236 22157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:35.991293 22159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:35.991240 22156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:35.991470 22151 server_base.cc:1034] running on GCE node
I20260811 05:12:35.992205 22151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:35.993326 22151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:35.994541 22151 hybrid_clock.cc:648] HybridClock initialized: now 1786425155994527 us; error 32 us; skew 500 ppm
Aug 11 05:12:35 dist-test-slave-7c35 krb5kdc[22137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:35.997314 22151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:35.998205 22151 webserver.cc:460] Webserver started at http://127.0.0.1:34481/ using document root <none> and password file <none>
I20260811 05:12:35.998781 22151 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:35.998899 22151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:35.999140 22151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:36.001187 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "c98d9257b97c4cee9d41262e08383009"
format_stamp: "Formatted at 2026-08-11 05:12:35 on dist-test-slave-7c35"
server_key: "de6097788418191d85ceed6836fcfaf1"
server_key_iv: "f776d007d3d36f6b36e4403d4ee7df92"
server_key_version: "encryptionkey@0"
I20260811 05:12:36.001771 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "c98d9257b97c4cee9d41262e08383009"
format_stamp: "Formatted at 2026-08-11 05:12:35 on dist-test-slave-7c35"
server_key: "de6097788418191d85ceed6836fcfaf1"
server_key_iv: "f776d007d3d36f6b36e4403d4ee7df92"
server_key_version: "encryptionkey@0"
I20260811 05:12:36.005314 22151 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:36.007681 22167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:36.008826 22151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:36.008975 22151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "c98d9257b97c4cee9d41262e08383009"
format_stamp: "Formatted at 2026-08-11 05:12:35 on dist-test-slave-7c35"
server_key: "de6097788418191d85ceed6836fcfaf1"
server_key_iv: "f776d007d3d36f6b36e4403d4ee7df92"
server_key_version: "encryptionkey@0"
I20260811 05:12:36.009080 22151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:36.019802 22151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:36.022405 22151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:36.022600 22151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:36.029943 22151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43807
I20260811 05:12:36.029959 22227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43807 every 8 connection(s)
I20260811 05:12:36.030828 22151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:36.033418 22228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:36.034273 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22151
I20260811 05:12:36.034411 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:36.034710 31075 external_mini_cluster.cc:1428] Setting key f44abd52ae323337afe4c7421cd6d0db
I20260811 05:12:36.039459 22228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009: Bootstrap starting.
I20260811 05:12:36.041775 22228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:36.042502 22228 log.cc:824] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:36.043713 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:36.036111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51967 (local address 127.0.0.1:43807)
0811 05:12:36.036535 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:36.036549 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:36.036582 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:12:36.037200 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:36.037216 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:36.037520 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:36.037840 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:36.038255 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:36.039296 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:36.040256 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:36.040718 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:36.040889 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:36.041822 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:36.041827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:36.041841 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:36.042258 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:36.042308 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:36.042339 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:36.042490 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:36.042674 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 05:12:36.042814 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":133,"threads_started":1}
I20260811 05:12:36.044524 22228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009: No bootstrap required, opened a new log
I20260811 05:12:36.046489 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:36.048298 22228 raft_consensus.cc:348] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 05:12:36.048555 22228 raft_consensus.cc:374] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:36.048630 22228 raft_consensus.cc:729] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98d9257b97c4cee9d41262e08383009, State: Initialized, Role: FOLLOWER
I20260811 05:12:36.049108 22228 consensus_queue.cc:260] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 05:12:36.049273 22228 raft_consensus.cc:388] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:36.049365 22228 raft_consensus.cc:482] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:36.049477 22228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:36.050593 22228 raft_consensus.cc:504] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 05:12:36.050997 22228 leader_election.cc:302] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c98d9257b97c4cee9d41262e08383009; no voters:
I20260811 05:12:36.051349 22228 leader_election.cc:288] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:36.051431 22239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:36.051612 22239 raft_consensus.cc:686] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [term 1 LEADER]: Becoming Leader. State: Replica: c98d9257b97c4cee9d41262e08383009, State: Running, Role: LEADER
I20260811 05:12:36.052238 22239 consensus_queue.cc:237] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } }
I20260811 05:12:36.052663 22228 sys_catalog.cc:564] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:36.053012 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:36.048622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40577 (local address 127.0.0.1:43807)
0811 05:12:36.048777 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:36.048781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:36.048860 (+ 79us) server_negotiation.cc:407] Connection header received
0811 05:12:36.049006 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:36.049009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:36.049057 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:36.049138 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:36.049453 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:36.050221 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:36.051183 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:36.051490 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:36.051595 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:36.052489 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:36.052493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:36.052495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:36.052508 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:36.052539 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:36.052550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:36.052648 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:36.052832 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 05:12:36.052867 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 05:12:36.053449 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c98d9257b97c4cee9d41262e08383009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } } }
I20260811 05:12:36.053646 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:36.053853 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c98d9257b97c4cee9d41262e08383009. Latest consensus state: current_term: 1 leader_uuid: "c98d9257b97c4cee9d41262e08383009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98d9257b97c4cee9d41262e08383009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43807 } } }
I20260811 05:12:36.053944 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:36.054520 22250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:36.057672 22250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:36.063357 22250 catalog_manager.cc:1347] Generated new cluster ID: bc65cd399b5844979a17e209f742ef31
I20260811 05:12:36.063457 22250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:36.081146 22250 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:36.082268 22250 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:36.090036 22250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c98d9257b97c4cee9d41262e08383009: Generated new TSK 0
I20260811 05:12:36.090726 22250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:36.098357 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22151
W20260811 05:12:36.107345 22135 connection.cc:535] client connection to 127.0.0.1:43807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43807): Connection reset by peer (error 104)
2026-08-11T05:12:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:36 dist-test-slave-7c35 krb5kdc[22264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:36 dist-test-slave-7c35 krb5kdc[22264](info): set up 2 sockets
Aug 11 05:12:36 dist-test-slave-7c35 krb5kdc[22264](info): commencing operation
krb5kdc: starting...
W20260811 05:12:37.182492 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:37 dist-test-slave-7c35 krb5kdc[22264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:37.303843 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:37.410496 22278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:37.410749 22278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:37.410817 22278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:37.414508 22278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:37.414584 22278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:37.414618 22278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:37.414669 22278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:37.414700 22278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:37.419472 22278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:37.420395 22278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:37.421362 22278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:37.429430 22285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:37.429459 22283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:37.429523 22287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:37.429654 22278 server_base.cc:1034] running on GCE node
I20260811 05:12:37.430276 22278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:37.431326 22278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:37.432510 22278 hybrid_clock.cc:648] HybridClock initialized: now 1786425157432485 us; error 36 us; skew 500 ppm
Aug 11 05:12:37 dist-test-slave-7c35 krb5kdc[22264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:37.435096 22278 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:37.435846 22278 webserver.cc:460] Webserver started at http://127.0.0.1:41553/ using document root <none> and password file <none>
I20260811 05:12:37.436334 22278 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:37.436420 22278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:37.436637 22278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:37.438599 22278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "79c9af055c8442f6becc0448ea3d8637"
format_stamp: "Formatted at 2026-08-11 05:12:37 on dist-test-slave-7c35"
server_key: "97d1ec5bfb6ee51fe4da739459fbb334"
server_key_iv: "d7bad38c4c2475607fd4cf7fee8a9bc7"
server_key_version: "encryptionkey@0"
I20260811 05:12:37.439275 22278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "79c9af055c8442f6becc0448ea3d8637"
format_stamp: "Formatted at 2026-08-11 05:12:37 on dist-test-slave-7c35"
server_key: "97d1ec5bfb6ee51fe4da739459fbb334"
server_key_iv: "d7bad38c4c2475607fd4cf7fee8a9bc7"
server_key_version: "encryptionkey@0"
I20260811 05:12:37.443444 22278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:12:37.445757 22294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:37.447079 22278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:37.447233 22278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "79c9af055c8442f6becc0448ea3d8637"
format_stamp: "Formatted at 2026-08-11 05:12:37 on dist-test-slave-7c35"
server_key: "97d1ec5bfb6ee51fe4da739459fbb334"
server_key_iv: "d7bad38c4c2475607fd4cf7fee8a9bc7"
server_key_version: "encryptionkey@0"
I20260811 05:12:37.447341 22278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:37.451644 22278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:37.454296 22278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:37.454470 22278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:37.461194 22278 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38527
I20260811 05:12:37.461258 22356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38527 every 8 connection(s)
I20260811 05:12:37.462042 22278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:37.464670 22357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:37.469328 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22278
I20260811 05:12:37.469480 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:37.469775 31075 external_mini_cluster.cc:1428] Setting key bdfbc671d144cf35cef059be73d1991e
I20260811 05:12:37.469879 22357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637: Bootstrap starting.
I20260811 05:12:37.472613 22357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:37.473467 22357 log.cc:824] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:37.475291 22357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637: No bootstrap required, opened a new log
I20260811 05:12:37.478929 22357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260811 05:12:37.479172 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:37.471089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48549 (local address 127.0.0.1:38527)
0811 05:12:37.471517 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:37.471533 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:37.471571 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:12:37.472377 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:37.472400 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:37.472741 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:37.473102 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:37.477722 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:37.477731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:37.477750 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:37.477766 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:37.477839 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:37.477884 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:37.478037 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:37.478121 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 05:12:37.478259 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":103,"threads_started":1}
I20260811 05:12:37.479173 22357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:37.479422 22357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c9af055c8442f6becc0448ea3d8637, State: Initialized, Role: FOLLOWER
I20260811 05:12:37.479792 22357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260811 05:12:37.479912 22357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:37.479988 22357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:37.480113 22357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:37.480952 22357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260811 05:12:37.481314 22357 leader_election.cc:302] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79c9af055c8442f6becc0448ea3d8637; no voters:
I20260811 05:12:37.481669 22357 leader_election.cc:288] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:37.481769 22362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:37.481848 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:37.481966 22362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [term 1 LEADER]: Becoming Leader. State: Replica: 79c9af055c8442f6becc0448ea3d8637, State: Running, Role: LEADER
I20260811 05:12:37.482234 22362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260811 05:12:37.482802 22357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:37.483357 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79c9af055c8442f6becc0448ea3d8637. Latest consensus state: current_term: 1 leader_uuid: "79c9af055c8442f6becc0448ea3d8637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260811 05:12:37.483476 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:37.483887 22372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:37.483865 22364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79c9af055c8442f6becc0448ea3d8637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c9af055c8442f6becc0448ea3d8637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260811 05:12:37.484254 22364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:37.485857 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:37.484076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54545 (local address 127.0.0.1:38527)
0811 05:12:37.484170 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:37.484174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:37.484571 (+ 397us) server_negotiation.cc:407] Connection header received
0811 05:12:37.484702 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:37.484705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:37.484739 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:37.484806 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:37.485474 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:37.485477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:37.485480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:37.485490 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:37.485512 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:37.485524 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:37.485594 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:37.485694 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:12:37.485730 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:12:37.487123 22372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:37.491986 22372 catalog_manager.cc:1347] Generated new cluster ID: da92d74d38714bd29797c0b1a01c0333
I20260811 05:12:37.492053 22372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:37.503638 22372 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:37.503945 22372 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:37.511519 22372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79c9af055c8442f6becc0448ea3d8637: Generated new TSK 0
I20260811 05:12:37.512130 22372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:37.515687 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22278
2026-08-11T05:12:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:37 dist-test-slave-7c35 krb5kdc[22390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:37 dist-test-slave-7c35 krb5kdc[22390](info): set up 2 sockets
Aug 11 05:12:37 dist-test-slave-7c35 krb5kdc[22390](info): commencing operation
krb5kdc: starting...
W20260811 05:12:38.580338 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:38 dist-test-slave-7c35 krb5kdc[22390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:38.705139 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:38.811085 22407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:38.811318 22407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:38.811388 22407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:38.815158 22407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:38.815204 22407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:38.815222 22407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:38.815263 22407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:38.815294 22407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:38.820232 22407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:38.821172 22407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:38.822090 22407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:38.830206 22414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:38.830262 22407 server_base.cc:1034] running on GCE node
W20260811 05:12:38.830222 22416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:38.830222 22413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:38.831172 22407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:38.832290 22407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:38.833494 22407 hybrid_clock.cc:648] HybridClock initialized: now 1786425158833472 us; error 40 us; skew 500 ppm
Aug 11 05:12:38 dist-test-slave-7c35 krb5kdc[22390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:38.836270 22407 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:38.837148 22407 webserver.cc:460] Webserver started at http://127.0.0.1:42299/ using document root <none> and password file <none>
I20260811 05:12:38.837672 22407 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:38.837754 22407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:38.838011 22407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:38.840413 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "9681bcf96e3343d0b73332f07a332369"
format_stamp: "Formatted at 2026-08-11 05:12:38 on dist-test-slave-7c35"
server_key: "11d11c61aec1204493dcee57b7ef60c2"
server_key_iv: "790f73734e7af9187e23298fc5754ed1"
server_key_version: "encryptionkey@0"
I20260811 05:12:38.841037 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "9681bcf96e3343d0b73332f07a332369"
format_stamp: "Formatted at 2026-08-11 05:12:38 on dist-test-slave-7c35"
server_key: "11d11c61aec1204493dcee57b7ef60c2"
server_key_iv: "790f73734e7af9187e23298fc5754ed1"
server_key_version: "encryptionkey@0"
I20260811 05:12:38.844901 22407 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:12:38.847255 22424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:38.848420 22407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:38.848587 22407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "9681bcf96e3343d0b73332f07a332369"
format_stamp: "Formatted at 2026-08-11 05:12:38 on dist-test-slave-7c35"
server_key: "11d11c61aec1204493dcee57b7ef60c2"
server_key_iv: "790f73734e7af9187e23298fc5754ed1"
server_key_version: "encryptionkey@0"
I20260811 05:12:38.848701 22407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:38.875046 22407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:38.878002 22407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:38.878239 22407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:38.886165 22407 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36163
I20260811 05:12:38.886173 22482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36163 every 8 connection(s)
I20260811 05:12:38.887197 22407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:38.889906 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:38.890762 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22407
I20260811 05:12:38.890925 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:38.891245 31075 external_mini_cluster.cc:1428] Setting key 3bfb364b84eb0a6eb9f6c47d9dc54ae8
I20260811 05:12:38.896122 22483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369: Bootstrap starting.
I20260811 05:12:38.898700 22483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:38.899665 22483 log.cc:824] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:12:38 dist-test-slave-7c35 krb5kdc[22390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:12:38.901433 22483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369: No bootstrap required, opened a new log
I20260811 05:12:38.904737 22483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260811 05:12:38.904975 22483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:38.905059 22483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9681bcf96e3343d0b73332f07a332369, State: Initialized, Role: FOLLOWER
I20260811 05:12:38.905426 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:38.892673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33305 (local address 127.0.0.1:36163)
0811 05:12:38.893084 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:38.893099 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:38.893143 (+ 44us) server_negotiation.cc:407] Connection header received
0811 05:12:38.893774 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:38.893790 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:38.894091 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:38.894344 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:38.895246 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:38.896135 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:38.897270 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:38.897731 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:38.897837 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:38.900887 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:38.900893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:38.900904 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:38.900929 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:38.902685 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:38.903151 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:38.903157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:38.903170 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:38.903271 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:38.903560 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:38.903563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:38.903565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:38.903935 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:38.904141 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:38.904374 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 05:12:38.904632 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":137,"threads_started":1}
I20260811 05:12:38.905557 22483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260811 05:12:38.905710 22483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:38.905773 22483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:38.905861 22483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:38.906755 22483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260811 05:12:38.907116 22483 leader_election.cc:302] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9681bcf96e3343d0b73332f07a332369; no voters:
I20260811 05:12:38.907446 22483 leader_election.cc:288] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:38.907512 22488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:38.907680 22488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [term 1 LEADER]: Becoming Leader. State: Replica: 9681bcf96e3343d0b73332f07a332369, State: Running, Role: LEADER
I20260811 05:12:38.907938 22488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } }
I20260811 05:12:38.908147 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:38.908578 22483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:38.909029 22490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9681bcf96e3343d0b73332f07a332369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } } }
I20260811 05:12:38.909061 22489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9681bcf96e3343d0b73332f07a332369. Latest consensus state: current_term: 1 leader_uuid: "9681bcf96e3343d0b73332f07a332369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9681bcf96e3343d0b73332f07a332369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36163 } } }
I20260811 05:12:38.909191 22489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:38.909144 22490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:38.909631 22494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:38.912906 22494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:38.918841 22494 catalog_manager.cc:1347] Generated new cluster ID: 1c3d30423f13409ab60938d644cc713a
I20260811 05:12:38.918944 22494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:38.920084 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:38.911007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44729 (local address 127.0.0.1:36163)
0811 05:12:38.911091 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:38.911094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:38.911682 (+ 588us) server_negotiation.cc:407] Connection header received
0811 05:12:38.911849 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:38.911852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:38.911912 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:38.912003 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:38.912816 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:38.913404 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:38.914570 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:38.914930 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:38.915017 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:38.916613 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:38.916617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:38.916619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:38.916641 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:38.918144 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:38.918610 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:38.918613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:38.918614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:38.918698 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:38.919146 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:38.919150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:38.919151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:38.919376 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:38.919512 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:38.919781 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 05:12:38.919913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:12:38.938537 22494 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:38.939584 22494 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:38.947755 22494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9681bcf96e3343d0b73332f07a332369: Generated new TSK 0
I20260811 05:12:38.948519 22494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:38.957412 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22407
2026-08-11T05:12:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:39 dist-test-slave-7c35 krb5kdc[22519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:39 dist-test-slave-7c35 krb5kdc[22519](info): set up 2 sockets
Aug 11 05:12:39 dist-test-slave-7c35 krb5kdc[22519](info): commencing operation
krb5kdc: starting...
W20260811 05:12:40.050587 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:40 dist-test-slave-7c35 krb5kdc[22519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:40.171265 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:40.277848 22533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:40.278069 22533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:40.278107 22533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:40.282029 22533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:40.282084 22533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:40.282104 22533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:40.282120 22533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:40.282135 22533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:40.287040 22533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:40.287851 22533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:40.288691 22533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:40.296756 22540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:40.296852 22538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:40.296861 22533 server_base.cc:1034] running on GCE node
W20260811 05:12:40.296756 22542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:40.297649 22533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:40.298694 22533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:40.299880 22533 hybrid_clock.cc:648] HybridClock initialized: now 1786425160299869 us; error 29 us; skew 500 ppm
Aug 11 05:12:40 dist-test-slave-7c35 krb5kdc[22519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:40.302603 22533 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:40.303490 22533 webserver.cc:460] Webserver started at http://127.0.0.1:38463/ using document root <none> and password file <none>
I20260811 05:12:40.304009 22533 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:40.304100 22533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:40.304316 22533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:40.306129 22533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "c9f0557d56134f1f8521acb162bc74ff"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-7c35"
server_key: "ffb263b2889e2ebecd237a9c5aba721f"
server_key_iv: "fa90bbd6c7cf5f774da68a47d302b19d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.306658 22533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "c9f0557d56134f1f8521acb162bc74ff"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-7c35"
server_key: "ffb263b2889e2ebecd237a9c5aba721f"
server_key_iv: "fa90bbd6c7cf5f774da68a47d302b19d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.309928 22533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:12:40.312160 22549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:40.313215 22533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:12:40.313345 22533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "c9f0557d56134f1f8521acb162bc74ff"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-7c35"
server_key: "ffb263b2889e2ebecd237a9c5aba721f"
server_key_iv: "fa90bbd6c7cf5f774da68a47d302b19d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.313442 22533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:40.333251 22533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:40.335914 22533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:40.336103 22533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:40.342878 22533 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39685
I20260811 05:12:40.342913 22614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39685 every 8 connection(s)
I20260811 05:12:40.343772 22533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:40.346349 22615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:40.347615 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22533
I20260811 05:12:40.347769 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:40.348093 31075 external_mini_cluster.cc:1428] Setting key d5984998a2b40494e70950b670905835
I20260811 05:12:40.352408 22615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff: Bootstrap starting.
I20260811 05:12:40.354749 22615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:40.355551 22615 log.cc:824] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:40.356815 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:40.349470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46389 (local address 127.0.0.1:39685)
0811 05:12:40.349897 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:40.349911 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:40.349943 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:12:40.350586 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:40.350601 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:40.350971 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:40.351279 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:40.351671 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.352698 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.353647 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.354117 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.354297 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:40.355217 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:40.355222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:40.355234 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:40.355599 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:40.355682 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:40.355721 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:40.355824 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:40.355971 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 05:12:40.356170 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":135,"threads_started":1}
I20260811 05:12:40.357543 22615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff: No bootstrap required, opened a new log
I20260811 05:12:40.359668 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:40.362180 22615 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 05:12:40.362385 22615 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:40.362427 22615 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f0557d56134f1f8521acb162bc74ff, State: Initialized, Role: FOLLOWER
I20260811 05:12:40.362953 22615 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 05:12:40.363131 22615 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:40.363202 22615 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:40.363301 22615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:40.364303 22615 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 05:12:40.364614 22615 leader_election.cc:302] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9f0557d56134f1f8521acb162bc74ff; no voters:
I20260811 05:12:40.364945 22615 leader_election.cc:288] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:40.365078 22626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:40.365302 22626 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [term 1 LEADER]: Becoming Leader. State: Replica: c9f0557d56134f1f8521acb162bc74ff, State: Running, Role: LEADER
I20260811 05:12:40.365628 22626 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 05:12:40.366317 22615 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:40.366570 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:40.362053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43261 (local address 127.0.0.1:39685)
0811 05:12:40.362187 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:40.362193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:40.362436 (+ 243us) server_negotiation.cc:407] Connection header received
0811 05:12:40.362586 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:40.362589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:40.362640 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:40.362735 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:40.363053 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.363808 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.364667 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.365044 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.365174 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:40.366148 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:40.366152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:40.366155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:40.366167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:40.366195 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:40.366207 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:40.366284 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:40.366399 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:12:40.366453 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:12:40.367334 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f0557d56134f1f8521acb162bc74ff. Latest consensus state: current_term: 1 leader_uuid: "c9f0557d56134f1f8521acb162bc74ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } } }
I20260811 05:12:40.367724 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:40.367616 22628 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f0557d56134f1f8521acb162bc74ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f0557d56134f1f8521acb162bc74ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } } }
I20260811 05:12:40.368055 22628 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:40.368983 22636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:40.371608 22636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:40.376691 22636 catalog_manager.cc:1347] Generated new cluster ID: 840e8665dd854a52be77ffca782bd733
I20260811 05:12:40.376753 22636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:40.385591 22636 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:40.386530 22636 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:40.391165 22636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9f0557d56134f1f8521acb162bc74ff: Generated new TSK 0
I20260811 05:12:40.391763 22636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:40.400131 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22533
2026-08-11T05:12:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:40 dist-test-slave-7c35 krb5kdc[22648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:40 dist-test-slave-7c35 krb5kdc[22648](info): set up 2 sockets
Aug 11 05:12:40 dist-test-slave-7c35 krb5kdc[22648](info): commencing operation
krb5kdc: starting...
W20260811 05:12:41.514699 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:41 dist-test-slave-7c35 krb5kdc[22648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:41.638330 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41991
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:41.756474 22664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:41.756866 22664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:41.756979 22664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:41.765249 22664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:41.765334 22664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:41.765383 22664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:41.765425 22664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:41.765455 22664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:41.773756 22664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:41.774784 22664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:41.775763 22664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:41.783484 22672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:41.783509 22669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:41.783644 22664 server_base.cc:1034] running on GCE node
W20260811 05:12:41.783489 22670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:41.784289 22664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:41.785300 22664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:41.786456 22664 hybrid_clock.cc:648] HybridClock initialized: now 1786425161786436 us; error 30 us; skew 500 ppm
Aug 11 05:12:41 dist-test-slave-7c35 krb5kdc[22648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:41.789108 22664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:41.789919 22664 webserver.cc:460] Webserver started at http://127.0.0.1:44113/ using document root <none> and password file <none>
I20260811 05:12:41.790462 22664 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:41.790524 22664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:41.790773 22664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:41.792714 22664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "e1f2597e65e940af88124cdba52b31a0"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-7c35"
server_key: "475a70d748376ee5acdd6bc4d9d420d5"
server_key_iv: "90d987f092bb654f709e51c22a2c727e"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.793262 22664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "e1f2597e65e940af88124cdba52b31a0"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-7c35"
server_key: "475a70d748376ee5acdd6bc4d9d420d5"
server_key_iv: "90d987f092bb654f709e51c22a2c727e"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.796729 22664 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:41.799026 22679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.800161 22664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:41.800295 22664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "e1f2597e65e940af88124cdba52b31a0"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-7c35"
server_key: "475a70d748376ee5acdd6bc4d9d420d5"
server_key_iv: "90d987f092bb654f709e51c22a2c727e"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.800397 22664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:41.804654 22664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:41.807830 22664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:41.808012 22664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:41.814540 22743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41991 every 8 connection(s)
I20260811 05:12:41.814548 22664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41991
I20260811 05:12:41.815344 22664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:41.817827 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:41.822821 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0: Bootstrap starting.
I20260811 05:12:41.824738 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22664
I20260811 05:12:41.824877 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:41.825071 22744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:41.825212 31075 external_mini_cluster.cc:1428] Setting key 6d705afd621d44cf86f741eef3fe0aff
I20260811 05:12:41.825824 22744 log.cc:824] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:41.827813 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0: No bootstrap required, opened a new log
I20260811 05:12:41.829818 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.826162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45453 (local address 127.0.0.1:41991)
0811 05:12:41.826561 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.826579 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.826621 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:12:41.827347 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.827365 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.827612 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.827909 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.828682 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.828686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.828696 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:41.828705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.828750 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:41.828766 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.828845 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.829008 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.829165 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260811 05:12:41.832002 22744 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } }
I20260811 05:12:41.832325 22744 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:41.832396 22744 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f2597e65e940af88124cdba52b31a0, State: Initialized, Role: FOLLOWER
I20260811 05:12:41.832751 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:41.832810 22744 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } }
I20260811 05:12:41.832990 22744 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:41.833056 22744 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:41.833160 22744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:41.833966 22744 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } }
I20260811 05:12:41.834321 22744 leader_election.cc:302] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f2597e65e940af88124cdba52b31a0; no voters:
I20260811 05:12:41.834576 22744 leader_election.cc:288] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:41.834733 22754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:41.834987 22754 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [term 1 LEADER]: Becoming Leader. State: Replica: e1f2597e65e940af88124cdba52b31a0, State: Running, Role: LEADER
I20260811 05:12:41.835304 22754 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } }
I20260811 05:12:41.835878 22744 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:41.836689 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.834908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44319 (local address 127.0.0.1:41991)
0811 05:12:41.835004 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.835009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.835234 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:12:41.835393 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.835395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.835423 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.835467 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.836241 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.836244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.836246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:41.836256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.836279 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:41.836290 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.836341 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.836518 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.836558 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:12:41.836895 22756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f2597e65e940af88124cdba52b31a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } } }
I20260811 05:12:41.837020 22756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:41.837394 22755 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f2597e65e940af88124cdba52b31a0. Latest consensus state: current_term: 1 leader_uuid: "e1f2597e65e940af88124cdba52b31a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f2597e65e940af88124cdba52b31a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41991 } } }
I20260811 05:12:41.837486 22755 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:41.837563 22766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:41.840437 22766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:41.845146 22766 catalog_manager.cc:1347] Generated new cluster ID: 322b94bd34c94b219432ebc29eb1b48c
I20260811 05:12:41.845218 22766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:41.859387 22766 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:41.859691 22766 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:41.865705 22766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f2597e65e940af88124cdba52b31a0: Generated new TSK 0
I20260811 05:12:41.866324 22766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:41.872455 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22664
2026-08-11T05:12:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:41 dist-test-slave-7c35 krb5kdc[22778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:41 dist-test-slave-7c35 krb5kdc[22778](info): set up 2 sockets
Aug 11 05:12:41 dist-test-slave-7c35 krb5kdc[22778](info): commencing operation
krb5kdc: starting...
W20260811 05:12:42.945653 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:43.067523 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:43.173908 22792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:43.174201 22792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:43.174278 22792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:43.178061 22792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:43.178157 22792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:43.178194 22792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:43.178229 22792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:43.178262 22792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:43.183732 22792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:43.184721 22792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:43.185734 22792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:43.193981 22797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.194031 22800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.194077 22792 server_base.cc:1034] running on GCE node
W20260811 05:12:43.194028 22798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.194823 22792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:43.195847 22792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:43.197036 22792 hybrid_clock.cc:648] HybridClock initialized: now 1786425163197020 us; error 27 us; skew 500 ppm
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:43.205118 22792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:43.205915 22792 webserver.cc:460] Webserver started at http://127.0.0.1:46091/ using document root <none> and password file <none>
I20260811 05:12:43.206368 22792 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:43.206420 22792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:43.206578 22792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:43.208425 22792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "23c8e26687074f01bea57c87208d1bb0"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-7c35"
server_key: "b9aa9b7476ade359a679bfd8c17015dd"
server_key_iv: "6207f6cead90afbe9e0a0c3bdda3cb42"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.208894 22792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "23c8e26687074f01bea57c87208d1bb0"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-7c35"
server_key: "b9aa9b7476ade359a679bfd8c17015dd"
server_key_iv: "6207f6cead90afbe9e0a0c3bdda3cb42"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.212296 22792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 05:12:43.214501 22809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.215687 22792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:43.215782 22792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "23c8e26687074f01bea57c87208d1bb0"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-7c35"
server_key: "b9aa9b7476ade359a679bfd8c17015dd"
server_key_iv: "6207f6cead90afbe9e0a0c3bdda3cb42"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.215859 22792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:43.230674 22792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:43.233094 22792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:43.233218 22792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:43.239640 22792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43427
I20260811 05:12:43.239651 22868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43427 every 8 connection(s)
I20260811 05:12:43.240496 22792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:43.243091 22870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:43.243180 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22792
I20260811 05:12:43.243369 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:43.243655 31075 external_mini_cluster.cc:1428] Setting key 9380b15e5c87c9738c5395f2eb5a3ff7
I20260811 05:12:43.249801 22870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0: Bootstrap starting.
I20260811 05:12:43.252142 22870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0: Neither blocks nor log segments found. Creating new log.
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:12:43.252915 22870 log.cc:824] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:43.255056 22870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0: No bootstrap required, opened a new log
I20260811 05:12:43.258066 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.244966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:60747 (local address 127.0.0.1:43427)
0811 05:12:43.245366 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.245383 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.245431 (+ 48us) server_negotiation.cc:407] Connection header received
0811 05:12:43.246232 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.246256 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.246646 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.247029 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.247684 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.249168 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.250015 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.250681 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.250832 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.253289 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.253295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.253316 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.253337 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.255123 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.255569 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.255575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.255580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.255706 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.255991 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.255994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.255995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.256435 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.256652 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.256884 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.257128 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260811 05:12:43.259410 22870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } }
I20260811 05:12:43.259608 22870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:43.259665 22870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c8e26687074f01bea57c87208d1bb0, State: Initialized, Role: FOLLOWER
I20260811 05:12:43.260129 22870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } }
I20260811 05:12:43.260329 22870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:43.260401 22870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:43.260491 22870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:43.261127 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:43.261405 22870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } }
I20260811 05:12:43.261708 22870 leader_election.cc:302] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23c8e26687074f01bea57c87208d1bb0; no voters:
I20260811 05:12:43.262007 22870 leader_election.cc:288] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:43.262071 22877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:43.262282 22877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [term 1 LEADER]: Becoming Leader. State: Replica: 23c8e26687074f01bea57c87208d1bb0, State: Running, Role: LEADER
I20260811 05:12:43.262583 22877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } }
I20260811 05:12:43.263202 22870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:43.263773 22877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23c8e26687074f01bea57c87208d1bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } } }
I20260811 05:12:43.263943 22877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:43.264190 22877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23c8e26687074f01bea57c87208d1bb0. Latest consensus state: current_term: 1 leader_uuid: "23c8e26687074f01bea57c87208d1bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23c8e26687074f01bea57c87208d1bb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43427 } } }
I20260811 05:12:43.264333 22877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:43.264642 22891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:43.268080 22891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:43.272202 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.263187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:49075 (local address 127.0.0.1:43427)
0811 05:12:43.263290 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.263294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.263520 (+ 226us) server_negotiation.cc:407] Connection header received
0811 05:12:43.263719 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.263721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.263763 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.263809 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.264735 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.265443 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.266615 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.266838 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.266963 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.268622 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.268626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.268629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.268657 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.270222 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.270781 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.270784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.270786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.270908 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.271316 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.271319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.271321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.271507 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.271645 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.271951 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.272049 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":43}
I20260811 05:12:43.274103 22891 catalog_manager.cc:1347] Generated new cluster ID: 6b6b6f2407024b7f8275b5c858cd4775
I20260811 05:12:43.274163 22891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:43.294313 22891 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:43.295455 22891 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:43.309789 22891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23c8e26687074f01bea57c87208d1bb0: Generated new TSK 0
I20260811 05:12:43.310422 22891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:43.316339 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22792
W20260811 05:12:43.325302 22776 connection.cc:535] client connection to 127.0.0.1:43427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43427): Connection reset by peer (error 104)
2026-08-11T05:12:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22905](info): set up 2 sockets
Aug 11 05:12:43 dist-test-slave-7c35 krb5kdc[22905](info): commencing operation
krb5kdc: starting...
W20260811 05:12:44.418761 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:44 dist-test-slave-7c35 krb5kdc[22905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:44.537919 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:44.646081 22921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:44.646309 22921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:44.646346 22921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:44.650151 22921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:44.650202 22921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:44.650221 22921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:44.650238 22921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:44.650252 22921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:44.655220 22921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:44.656052 22921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:44.656991 22921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:44.665063 22932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:44.665246 22921 server_base.cc:1034] running on GCE node
W20260811 05:12:44.665127 22927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:44.665071 22929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:44.665908 22921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:44.666814 22921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:44.668015 22921 hybrid_clock.cc:648] HybridClock initialized: now 1786425164667995 us; error 32 us; skew 500 ppm
Aug 11 05:12:44 dist-test-slave-7c35 krb5kdc[22905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:44.671020 22921 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:44.671880 22921 webserver.cc:460] Webserver started at http://127.0.0.1:42061/ using document root <none> and password file <none>
I20260811 05:12:44.672467 22921 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:44.672569 22921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:44.672792 22921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:44.674721 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "1e64ee4a593c4f92932cfcd0600764a3"
format_stamp: "Formatted at 2026-08-11 05:12:44 on dist-test-slave-7c35"
server_key: "a8043fc06ee98f25923474ca1b3b6804"
server_key_iv: "d1cbe78ef48a1259971f39b6e40f9e64"
server_key_version: "encryptionkey@0"
I20260811 05:12:44.675357 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "1e64ee4a593c4f92932cfcd0600764a3"
format_stamp: "Formatted at 2026-08-11 05:12:44 on dist-test-slave-7c35"
server_key: "a8043fc06ee98f25923474ca1b3b6804"
server_key_iv: "d1cbe78ef48a1259971f39b6e40f9e64"
server_key_version: "encryptionkey@0"
I20260811 05:12:44.678740 22921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:12:44.681015 22941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:44.682088 22921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:44.682224 22921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "1e64ee4a593c4f92932cfcd0600764a3"
format_stamp: "Formatted at 2026-08-11 05:12:44 on dist-test-slave-7c35"
server_key: "a8043fc06ee98f25923474ca1b3b6804"
server_key_iv: "d1cbe78ef48a1259971f39b6e40f9e64"
server_key_version: "encryptionkey@0"
I20260811 05:12:44.682327 22921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:44.699827 22921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:44.702346 22921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:44.702556 22921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:44.709178 22921 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34585
I20260811 05:12:44.709219 23002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34585 every 8 connection(s)
I20260811 05:12:44.710039 22921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:44.712594 23003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:44.713498 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 22921
I20260811 05:12:44.713645 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:44.713972 31075 external_mini_cluster.cc:1428] Setting key 822e15ea44c3a50fb81e5ee03111422e
I20260811 05:12:44.718499 23003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3: Bootstrap starting.
I20260811 05:12:44.721040 23003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:44.721870 23003 log.cc:824] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:44.722630 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:44.715349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:48509 (local address 127.0.0.1:34585)
0811 05:12:44.715729 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:44.715748 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:44.715779 (+ 31us) server_negotiation.cc:407] Connection header received
0811 05:12:44.716332 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:44.716347 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:44.716655 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:44.716908 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:44.717278 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:44.718203 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:44.719431 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:44.719864 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:44.720005 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:44.720836 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:44.720841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:44.720853 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:44.721264 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:44.721328 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:44.721351 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:44.721451 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:44.721659 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 05:12:44.721799 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260811 05:12:44.724013 23003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3: No bootstrap required, opened a new log
I20260811 05:12:44.725502 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:44.728386 23003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } }
I20260811 05:12:44.728668 23003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:44.728739 23003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e64ee4a593c4f92932cfcd0600764a3, State: Initialized, Role: FOLLOWER
I20260811 05:12:44.729115 23003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } }
I20260811 05:12:44.729254 23003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:44.729336 23003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:44.729435 23003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:44.730516 23003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } }
I20260811 05:12:44.730898 23003 leader_election.cc:302] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e64ee4a593c4f92932cfcd0600764a3; no voters:
I20260811 05:12:44.731230 23003 leader_election.cc:288] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:44.731338 23015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:44.731575 23015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1e64ee4a593c4f92932cfcd0600764a3, State: Running, Role: LEADER
I20260811 05:12:44.731900 23015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } }
I20260811 05:12:44.732273 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:44.727762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:44773 (local address 127.0.0.1:34585)
0811 05:12:44.727888 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:44.727892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:44.728145 (+ 253us) server_negotiation.cc:407] Connection header received
0811 05:12:44.728246 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:44.728248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:44.728282 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:44.728343 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:44.728658 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:44.729314 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:44.730440 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:44.730735 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:44.730815 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:44.731735 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:44.731738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:44.731741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:44.731752 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:44.731777 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:44.731789 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:44.731845 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:44.732098 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 05:12:44.732134 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:12:44.732509 23003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:44.733685 23017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e64ee4a593c4f92932cfcd0600764a3. Latest consensus state: current_term: 1 leader_uuid: "1e64ee4a593c4f92932cfcd0600764a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } } }
I20260811 05:12:44.733783 23017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:44.734148 23024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:44.734107 23016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e64ee4a593c4f92932cfcd0600764a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e64ee4a593c4f92932cfcd0600764a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34585 } } }
I20260811 05:12:44.734256 23016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:44.736918 23024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:44.742314 23024 catalog_manager.cc:1347] Generated new cluster ID: f9437190824e4044a6c617db4214976a
I20260811 05:12:44.742379 23024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:44.781235 23024 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:44.782474 23024 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:44.798036 23024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e64ee4a593c4f92932cfcd0600764a3: Generated new TSK 0
I20260811 05:12:44.798796 23024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:44.809628 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 22921
W20260811 05:12:44.816987 22903 connection.cc:535] client connection to 127.0.0.1:34585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34585): Connection reset by peer (error 104)
2026-08-11T05:12:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:44 dist-test-slave-7c35 krb5kdc[23038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:44 dist-test-slave-7c35 krb5kdc[23038](info): set up 2 sockets
Aug 11 05:12:44 dist-test-slave-7c35 krb5kdc[23038](info): commencing operation
krb5kdc: starting...
W20260811 05:12:45.889061 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:45 dist-test-slave-7c35 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:46.013767 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:46.125491 23052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:46.125792 23052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:46.125859 23052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:46.129981 23052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:46.130061 23052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:46.130123 23052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:46.130179 23052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:46.130227 23052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:46.135401 23052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:46.136458 23052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:46.137481 23052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:12:46.146039 23052 server_base.cc:1034] running on GCE node
W20260811 05:12:46.146013 23057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:46.146064 23060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:46.146013 23058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:46.146709 23052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:46.147769 23052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:46.148959 23052 hybrid_clock.cc:648] HybridClock initialized: now 1786425166148949 us; error 27 us; skew 500 ppm
Aug 11 05:12:46 dist-test-slave-7c35 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:46.151741 23052 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:46.152655 23052 webserver.cc:460] Webserver started at http://127.0.0.1:38529/ using document root <none> and password file <none>
I20260811 05:12:46.153204 23052 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:46.153295 23052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:46.153553 23052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:46.155638 23052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "b5fe6e8f99dc4eed906db434e3f4ed27"
format_stamp: "Formatted at 2026-08-11 05:12:46 on dist-test-slave-7c35"
server_key: "4965e9adf958749914f5bb74b340364b"
server_key_iv: "d3b6b971328928a08d021675dd571dbf"
server_key_version: "encryptionkey@0"
I20260811 05:12:46.156224 23052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "b5fe6e8f99dc4eed906db434e3f4ed27"
format_stamp: "Formatted at 2026-08-11 05:12:46 on dist-test-slave-7c35"
server_key: "4965e9adf958749914f5bb74b340364b"
server_key_iv: "d3b6b971328928a08d021675dd571dbf"
server_key_version: "encryptionkey@0"
I20260811 05:12:46.159915 23052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:12:46.162190 23068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:46.163345 23052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:46.163494 23052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "b5fe6e8f99dc4eed906db434e3f4ed27"
format_stamp: "Formatted at 2026-08-11 05:12:46 on dist-test-slave-7c35"
server_key: "4965e9adf958749914f5bb74b340364b"
server_key_iv: "d3b6b971328928a08d021675dd571dbf"
server_key_version: "encryptionkey@0"
I20260811 05:12:46.163614 23052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:46.167867 23052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:46.170753 23052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:46.170982 23052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:46.178013 23052 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42323
I20260811 05:12:46.178025 23130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42323 every 8 connection(s)
I20260811 05:12:46.178951 23052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:46.180271 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23052
I20260811 05:12:46.180421 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:46.180779 31075 external_mini_cluster.cc:1428] Setting key 634fc387d3725eb33edf915e996a1c61
I20260811 05:12:46.182313 23131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.186017 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.182125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:51107 (local address 127.0.0.1:42323)
0811 05:12:46.182554 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.182569 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.182603 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:12:46.183279 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.183298 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.183569 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.183845 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:46.184708 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:46.184713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:46.184724 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:46.184735 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:46.184787 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:46.184805 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:46.184960 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.185105 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.185284 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":139,"threads_started":1}
I20260811 05:12:46.188889 23131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27: Bootstrap starting.
I20260811 05:12:46.189178 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:46.191293 23131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.192219 23131 log.cc:824] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:46.192942 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.191299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:33729 (local address 127.0.0.1:42323)
0811 05:12:46.191435 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.191439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.191593 (+ 154us) server_negotiation.cc:407] Connection header received
0811 05:12:46.191666 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.191669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.191704 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.191786 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:46.192507 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:46.192511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:46.192513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:46.192525 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:46.192553 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:46.192572 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:46.192663 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.192765 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.192798 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:12:46.194257 23131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27: No bootstrap required, opened a new log
I20260811 05:12:46.197678 23131 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } }
I20260811 05:12:46.197912 23131 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.197995 23131 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fe6e8f99dc4eed906db434e3f4ed27, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.198422 23131 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } }
I20260811 05:12:46.198585 23131 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:46.198673 23131 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:46.198770 23131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.199652 23131 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } }
I20260811 05:12:46.199980 23131 leader_election.cc:302] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5fe6e8f99dc4eed906db434e3f4ed27; no voters:
I20260811 05:12:46.200273 23131 leader_election.cc:288] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:46.200392 23143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:46.200636 23143 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [term 1 LEADER]: Becoming Leader. State: Replica: b5fe6e8f99dc4eed906db434e3f4ed27, State: Running, Role: LEADER
I20260811 05:12:46.200984 23143 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } }
I20260811 05:12:46.201628 23131 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:46.202807 23144 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5fe6e8f99dc4eed906db434e3f4ed27. Latest consensus state: current_term: 1 leader_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } } }
I20260811 05:12:46.202818 23145 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fe6e8f99dc4eed906db434e3f4ed27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42323 } } }
I20260811 05:12:46.203016 23145 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:46.203284 23144 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:46.203464 23153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:46.206071 23153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:46.211668 23153 catalog_manager.cc:1347] Generated new cluster ID: 47a0cb59943f447db11b1887f4a846a4
I20260811 05:12:46.211732 23153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:46.230472 23153 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:46.230830 23153 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:46.236236 23153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5fe6e8f99dc4eed906db434e3f4ed27: Generated new TSK 0
I20260811 05:12:46.236881 23153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:46.241712 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23052
2026-08-11T05:12:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:46 dist-test-slave-7c35 krb5kdc[23165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:46 dist-test-slave-7c35 krb5kdc[23165](info): set up 2 sockets
Aug 11 05:12:46 dist-test-slave-7c35 krb5kdc[23165](info): commencing operation
krb5kdc: starting...
W20260811 05:12:47.345614 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:47.476030 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:47.588539 23181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:47.588783 23181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:47.588822 23181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:47.592898 23181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:47.592949 23181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:47.592968 23181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:47.593011 23181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:47.593042 23181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:47.598130 23181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:47.599184 23181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:47.600206 23181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:47.608870 23188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:47.608812 23190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:47.608956 23181 server_base.cc:1034] running on GCE node
W20260811 05:12:47.608814 23192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:47.609648 23181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:47.610689 23181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:47.611838 23181 hybrid_clock.cc:648] HybridClock initialized: now 1786425167611817 us; error 32 us; skew 500 ppm
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:47.614761 23181 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:47.615700 23181 webserver.cc:460] Webserver started at http://127.0.0.1:39841/ using document root <none> and password file <none>
I20260811 05:12:47.616263 23181 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:47.616356 23181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:47.616578 23181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:47.618649 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "8cf2a48893ab4304ba4353bf03b639bf"
format_stamp: "Formatted at 2026-08-11 05:12:47 on dist-test-slave-7c35"
server_key: "d9d2437cee8f06f2d699415618cedc56"
server_key_iv: "e0f54105c710ffad942fb54503e79a78"
server_key_version: "encryptionkey@0"
I20260811 05:12:47.619339 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "8cf2a48893ab4304ba4353bf03b639bf"
format_stamp: "Formatted at 2026-08-11 05:12:47 on dist-test-slave-7c35"
server_key: "d9d2437cee8f06f2d699415618cedc56"
server_key_iv: "e0f54105c710ffad942fb54503e79a78"
server_key_version: "encryptionkey@0"
I20260811 05:12:47.624166 23181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:12:47.627022 23199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:47.628341 23181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:12:47.628510 23181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "8cf2a48893ab4304ba4353bf03b639bf"
format_stamp: "Formatted at 2026-08-11 05:12:47 on dist-test-slave-7c35"
server_key: "d9d2437cee8f06f2d699415618cedc56"
server_key_iv: "e0f54105c710ffad942fb54503e79a78"
server_key_version: "encryptionkey@0"
I20260811 05:12:47.628640 23181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:47.642671 23181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:47.645659 23181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:47.645886 23181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:47.653740 23181 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35605
I20260811 05:12:47.653772 23257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35605 every 8 connection(s)
I20260811 05:12:47.654699 23181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:47.657430 23258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:47.662156 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23181
I20260811 05:12:47.662299 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:47.662648 31075 external_mini_cluster.cc:1428] Setting key f3f86956c4a52cd8fcb36b7c32e4f67c
I20260811 05:12:47.663043 23258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf: Bootstrap starting.
I20260811 05:12:47.665668 23258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:47.666438 23258 log.cc:824] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:47.668694 23258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf: No bootstrap required, opened a new log
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:12:47.672628 23258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260811 05:12:47.672931 23258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:47.672999 23258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf2a48893ab4304ba4353bf03b639bf, State: Initialized, Role: FOLLOWER
I20260811 05:12:47.673527 23258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260811 05:12:47.673705 23258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:47.673789 23258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:47.673892 23258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:47.675050 23258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260811 05:12:47.675423 23258 leader_election.cc:302] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cf2a48893ab4304ba4353bf03b639bf; no voters:
I20260811 05:12:47.675761 23258 leader_election.cc:288] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:47.675830 23263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:47.676054 23263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [term 1 LEADER]: Becoming Leader. State: Replica: 8cf2a48893ab4304ba4353bf03b639bf, State: Running, Role: LEADER
I20260811 05:12:47.676355 23263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260811 05:12:47.677029 23258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:47.678284 23266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cf2a48893ab4304ba4353bf03b639bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } } }
I20260811 05:12:47.678334 23267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cf2a48893ab4304ba4353bf03b639bf. Latest consensus state: current_term: 1 leader_uuid: "8cf2a48893ab4304ba4353bf03b639bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf2a48893ab4304ba4353bf03b639bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } } }
I20260811 05:12:47.678469 23266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:47.678706 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:47.664088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:52457 (local address 127.0.0.1:35605)
0811 05:12:47.664543 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:47.664559 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:47.664599 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:12:47.665307 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:47.665324 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:47.665714 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:47.666116 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:47.667111 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:47.668503 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:47.669429 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:47.670121 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:47.670286 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:47.672786 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:47.672792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:47.672807 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:47.672838 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:47.675112 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:47.675726 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:47.675732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:47.675754 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:47.675932 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:47.676338 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:47.676342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:47.676345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:47.676899 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:47.677153 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:47.677407 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 05:12:47.677650 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":123,"threads_started":1}
I20260811 05:12:47.678930 23277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:47.679045 23267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:47.681859 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:47.682004 23277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:47.688156 23277 catalog_manager.cc:1347] Generated new cluster ID: ef2bdef6fb664b6a927bf1627a5317d0
I20260811 05:12:47.688216 23277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:47.693734 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:47.684164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:50797 (local address 127.0.0.1:35605)
0811 05:12:47.684295 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:47.684299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:47.684731 (+ 432us) server_negotiation.cc:407] Connection header received
0811 05:12:47.684843 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:47.684846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:47.684898 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:47.684988 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:47.685807 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:47.686599 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:47.687765 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:47.688140 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:47.688252 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:47.690098 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:47.690102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:47.690104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:47.690128 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:47.691798 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:47.692372 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:47.692375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:47.692377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:47.692461 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:47.692845 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:47.692848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:47.692850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:47.693061 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:47.693231 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:47.693496 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 05:12:47.693592 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:12:47.694538 23277 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:47.695456 23277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:47.699654 23277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cf2a48893ab4304ba4353bf03b639bf: Generated new TSK 0
I20260811 05:12:47.700271 23277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:47.708026 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23181
W20260811 05:12:47.718973 23163 connection.cc:535] client connection to 127.0.0.1:35605 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35605): Connection reset by peer (error 104)
2026-08-11T05:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23298](info): set up 2 sockets
Aug 11 05:12:47 dist-test-slave-7c35 krb5kdc[23298](info): commencing operation
krb5kdc: starting...
W20260811 05:12:48.824798 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:48 dist-test-slave-7c35 krb5kdc[23298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:48.953606 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:49.065589 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:49.065829 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:49.065928 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:49.070330 23314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:49.070398 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:49.070418 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:49.070490 23314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:49.070529 23314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:49.075768 23314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:49.076761 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:49.077746 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:49.086309 23322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:49.086370 23319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:49.086316 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:49.086593 23314 server_base.cc:1034] running on GCE node
I20260811 05:12:49.087190 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:49.088341 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:49.089529 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786425169089508 us; error 41 us; skew 500 ppm
Aug 11 05:12:49 dist-test-slave-7c35 krb5kdc[23298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:49.096549 23314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:49.097486 23314 webserver.cc:460] Webserver started at http://127.0.0.1:34051/ using document root <none> and password file <none>
I20260811 05:12:49.098048 23314 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:49.098138 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:49.098366 23314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:49.100440 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "ae9101075aae4a78a93f5615ade6d928"
format_stamp: "Formatted at 2026-08-11 05:12:49 on dist-test-slave-7c35"
server_key: "d0dd4f3c30ed97d138094a44d50f8c62"
server_key_iv: "ba0e0c20cfde8186faccaf2e2bba67c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:49.101024 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "ae9101075aae4a78a93f5615ade6d928"
format_stamp: "Formatted at 2026-08-11 05:12:49 on dist-test-slave-7c35"
server_key: "d0dd4f3c30ed97d138094a44d50f8c62"
server_key_iv: "ba0e0c20cfde8186faccaf2e2bba67c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:49.104694 23314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:12:49.107074 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:49.108188 23314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:49.108340 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "ae9101075aae4a78a93f5615ade6d928"
format_stamp: "Formatted at 2026-08-11 05:12:49 on dist-test-slave-7c35"
server_key: "d0dd4f3c30ed97d138094a44d50f8c62"
server_key_iv: "ba0e0c20cfde8186faccaf2e2bba67c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:49.108449 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:49.129575 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:49.132560 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:49.132776 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:49.140040 23314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36125
I20260811 05:12:49.140069 23386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36125 every 8 connection(s)
I20260811 05:12:49.140993 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:49.143728 23387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.149365 23387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928: Bootstrap starting.
I20260811 05:12:49.150307 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23314
I20260811 05:12:49.150508 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:49.150807 31075 external_mini_cluster.cc:1428] Setting key faf765161ac7bdfb1223606eff25a648
I20260811 05:12:49.152122 23387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.152899 23387 log.cc:824] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:49.154961 23387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928: No bootstrap required, opened a new log
I20260811 05:12:49.158628 23387 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260811 05:12:49.158898 23387 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.158978 23387 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9101075aae4a78a93f5615ade6d928, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.159497 23387 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260811 05:12:49.159667 23387 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:49.159734 23387 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:49.159832 23387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.160200 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.151996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:41503 (local address 127.0.0.1:36125)
0811 05:12:49.152442 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.152457 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.152491 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:12:49.153212 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.153231 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.153675 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.154040 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:49.154459 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.155558 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.156474 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.156971 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.157142 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.158187 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:49.158193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:49.158206 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:49.158598 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:49.158654 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:49.158677 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:49.158823 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.159037 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.159180 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
I20260811 05:12:49.160962 23387 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260811 05:12:49.161309 23387 leader_election.cc:302] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae9101075aae4a78a93f5615ade6d928; no voters:
I20260811 05:12:49.161691 23387 leader_election.cc:288] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:49.161774 23392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:49.161957 23392 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [term 1 LEADER]: Becoming Leader. State: Replica: ae9101075aae4a78a93f5615ade6d928, State: Running, Role: LEADER
I20260811 05:12:49.162305 23392 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } }
I20260811 05:12:49.163036 23387 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:49.163566 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:49.164032 23393 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae9101075aae4a78a93f5615ade6d928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } } }
I20260811 05:12:49.164163 23393 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:49.164517 23394 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae9101075aae4a78a93f5615ade6d928. Latest consensus state: current_term: 1 leader_uuid: "ae9101075aae4a78a93f5615ade6d928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9101075aae4a78a93f5615ade6d928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36125 } } }
I20260811 05:12:49.164604 23394 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:49.164863 23403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:49.168318 23403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:49.171022 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.166109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:37359 (local address 127.0.0.1:36125)
0811 05:12:49.166229 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.166234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.166460 (+ 226us) server_negotiation.cc:407] Connection header received
0811 05:12:49.166594 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.166597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.166660 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.166743 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:49.167123 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.167912 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.169062 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.169410 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.169522 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.170508 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:49.170512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:49.170514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:49.170527 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:49.170560 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:49.170571 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:49.170670 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.170793 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.170828 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 05:12:49.174407 23403 catalog_manager.cc:1347] Generated new cluster ID: 50a84c25425e480a94a212e7d906fbd0
I20260811 05:12:49.174477 23403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:49.183668 23403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:49.184645 23403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:49.197098 23403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae9101075aae4a78a93f5615ade6d928: Generated new TSK 0
I20260811 05:12:49.197904 23403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:49.209354 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23314
W20260811 05:12:49.220319 23296 connection.cc:535] client connection to 127.0.0.1:36125 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36125): Connection reset by peer (error 104)
2026-08-11T05:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:49 dist-test-slave-7c35 krb5kdc[23423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:49 dist-test-slave-7c35 krb5kdc[23423](info): set up 2 sockets
Aug 11 05:12:49 dist-test-slave-7c35 krb5kdc[23423](info): commencing operation
krb5kdc: starting...
W20260811 05:12:50.317864 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:50 dist-test-slave-7c35 krb5kdc[23423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:12:50.437773 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:50.545657 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:50.545885 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:50.545923 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:50.549750 23441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:50.549801 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:50.549820 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:50.549836 23441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:50.549850 23441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:50.554772 23441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34005
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:50.555739 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:50.556687 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:50.564669 23451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:50.564677 23449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:50.564677 23447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:50.564960 23441 server_base.cc:1034] running on GCE node
I20260811 05:12:50.565506 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:50.566642 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:50.567824 23441 hybrid_clock.cc:648] HybridClock initialized: now 1786425170567798 us; error 57 us; skew 500 ppm
Aug 11 05:12:50 dist-test-slave-7c35 krb5kdc[23423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:50.570580 23441 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:12:50.571405 23441 webserver.cc:460] Webserver started at http://127.0.0.1:39411/ using document root <none> and password file <none>
I20260811 05:12:50.571893 23441 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:50.571950 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:50.572108 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:50.574054 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "d9b43a503b2646be870f12394e5bf12b"
format_stamp: "Formatted at 2026-08-11 05:12:50 on dist-test-slave-7c35"
server_key: "41b84d6a116de619b0b55e401c44ed0d"
server_key_iv: "3b0f01e97bd501eed4089245b0b86aec"
server_key_version: "encryptionkey@0"
I20260811 05:12:50.574563 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "d9b43a503b2646be870f12394e5bf12b"
format_stamp: "Formatted at 2026-08-11 05:12:50 on dist-test-slave-7c35"
server_key: "41b84d6a116de619b0b55e401c44ed0d"
server_key_iv: "3b0f01e97bd501eed4089245b0b86aec"
server_key_version: "encryptionkey@0"
I20260811 05:12:50.578029 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:12:50.580283 23459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:50.581453 23441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:50.581596 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "d9b43a503b2646be870f12394e5bf12b"
format_stamp: "Formatted at 2026-08-11 05:12:50 on dist-test-slave-7c35"
server_key: "41b84d6a116de619b0b55e401c44ed0d"
server_key_iv: "3b0f01e97bd501eed4089245b0b86aec"
server_key_version: "encryptionkey@0"
I20260811 05:12:50.581702 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:50.586079 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:50.589159 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:50.589336 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:50.596258 23441 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34005
I20260811 05:12:50.596285 23516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34005 every 8 connection(s)
I20260811 05:12:50.597105 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:50.599710 23517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.603478 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23441
I20260811 05:12:50.603621 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:50.603927 31075 external_mini_cluster.cc:1428] Setting key 6b9267403b47cc339a9f746a366ec727
I20260811 05:12:50.605265 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b: Bootstrap starting.
I20260811 05:12:50.607586 23517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.608376 23517 log.cc:824] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:50.608994 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.605273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:35299 (local address 127.0.0.1:34005)
0811 05:12:50.605725 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:50.605747 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:50.605781 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:12:50.606370 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:50.606385 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:50.606668 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:50.607030 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:50.607821 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:50.607826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:50.607837 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:50.607848 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:50.607899 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:50.607918 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:50.608029 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:50.608159 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 05:12:50.608310 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":140,"threads_started":1}
I20260811 05:12:50.610256 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b: No bootstrap required, opened a new log
I20260811 05:12:50.611732 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:50.614499 23517 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260811 05:12:50.614796 23517 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.614885 23517 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b43a503b2646be870f12394e5bf12b, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.615356 23517 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260811 05:12:50.615456 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.613699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:40539 (local address 127.0.0.1:34005)
0811 05:12:50.613874 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:50.613880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:50.614156 (+ 276us) server_negotiation.cc:407] Connection header received
0811 05:12:50.614267 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:50.614269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:50.614296 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:50.614358 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:50.615077 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:50.615080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:50.615082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:50.615092 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:50.615115 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:50.615125 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:50.615193 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:50.615281 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 05:12:50.615309 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:12:50.615569 23517 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:50.615643 23517 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:50.615736 23517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:50.616719 23517 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260811 05:12:50.617051 23517 leader_election.cc:302] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b43a503b2646be870f12394e5bf12b; no voters:
I20260811 05:12:50.617367 23517 leader_election.cc:288] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:50.617442 23528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:50.617674 23528 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [term 1 LEADER]: Becoming Leader. State: Replica: d9b43a503b2646be870f12394e5bf12b, State: Running, Role: LEADER
I20260811 05:12:50.618032 23528 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } }
I20260811 05:12:50.618569 23517 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:50.619190 23528 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b43a503b2646be870f12394e5bf12b. Latest consensus state: current_term: 1 leader_uuid: "d9b43a503b2646be870f12394e5bf12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } } }
I20260811 05:12:50.619371 23528 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:50.619660 23529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b43a503b2646be870f12394e5bf12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b43a503b2646be870f12394e5bf12b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34005 } } }
I20260811 05:12:50.619766 23529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:50.619812 23537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:50.622874 23537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:50.628582 23537 catalog_manager.cc:1347] Generated new cluster ID: a7f5c221d6ca451798824bb53d1de813
I20260811 05:12:50.628643 23537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:50.640738 23537 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:50.641045 23537 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:50.648196 23537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b43a503b2646be870f12394e5bf12b: Generated new TSK 0
I20260811 05:12:50.648878 23537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:50.655784 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23441
2026-08-11T05:12:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:50 dist-test-slave-7c35 krb5kdc[23554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:50 dist-test-slave-7c35 krb5kdc[23554](info): set up 2 sockets
Aug 11 05:12:50 dist-test-slave-7c35 krb5kdc[23554](info): commencing operation
krb5kdc: starting...
W20260811 05:12:51.761448 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:51 dist-test-slave-7c35 krb5kdc[23554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:51.901417 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:33779
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:44737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:33779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:52.010785 23570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:52.011055 23570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:52.011096 23570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:52.014897 23570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:52.014945 23570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:52.014963 23570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:52.014979 23570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:52.014994 23570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:52.019963 23570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:44737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:33779
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:33779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:52.020889 23570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:52.021792 23570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:52.029820 23579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:52.029887 23576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:52.029835 23577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:52.030293 23570 server_base.cc:1034] running on GCE node
I20260811 05:12:52.030772 23570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:52.031832 23570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:52.033004 23570 hybrid_clock.cc:648] HybridClock initialized: now 1786425172032975 us; error 39 us; skew 500 ppm
Aug 11 05:12:52 dist-test-slave-7c35 krb5kdc[23554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:52.035758 23570 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:52.036628 23570 webserver.cc:460] Webserver started at http://127.30.88.254:40545/ using document root <none> and password file <none>
I20260811 05:12:52.037168 23570 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:52.037230 23570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:52.037496 23570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:52.039574 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "0903a134d5a7443984a9827f3106a874"
format_stamp: "Formatted at 2026-08-11 05:12:52 on dist-test-slave-7c35"
server_key: "b9baed074dd2ac759583d3e414e73b93"
server_key_iv: "fad999a1fc8308a833cb79678c5c1e38"
server_key_version: "encryptionkey@0"
I20260811 05:12:52.040140 23570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "0903a134d5a7443984a9827f3106a874"
format_stamp: "Formatted at 2026-08-11 05:12:52 on dist-test-slave-7c35"
server_key: "b9baed074dd2ac759583d3e414e73b93"
server_key_iv: "fad999a1fc8308a833cb79678c5c1e38"
server_key_version: "encryptionkey@0"
I20260811 05:12:52.043987 23570 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:12:52.046307 23587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:52.047569 23570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:52.047668 23570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "0903a134d5a7443984a9827f3106a874"
format_stamp: "Formatted at 2026-08-11 05:12:52 on dist-test-slave-7c35"
server_key: "b9baed074dd2ac759583d3e414e73b93"
server_key_iv: "fad999a1fc8308a833cb79678c5c1e38"
server_key_version: "encryptionkey@0"
I20260811 05:12:52.047749 23570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:52.073421 23570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:52.076867 23570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:52.077105 23570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:52.085038 23570 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:33779
I20260811 05:12:52.085069 23649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:33779 every 8 connection(s)
I20260811 05:12:52.086073 23570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:52.087423 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23570
I20260811 05:12:52.087590 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:52.087920 31075 external_mini_cluster.cc:1428] Setting key 9390c72d67f8865fbfa9f9ce3ecd11b9
I20260811 05:12:52.089249 23650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:52.095857 23650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874: Bootstrap starting.
Aug 11 05:12:52 dist-test-slave-7c35 krb5kdc[23554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:52.098320 23650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:52.099184 23650 log.cc:824] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:52.101155 23650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874: No bootstrap required, opened a new log
I20260811 05:12:52.103338 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:52.089430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43607 (local address 127.30.88.254:33779)
0811 05:12:52.089870 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:52.089889 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:52.089930 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:12:52.090794 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:52.090819 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:52.091266 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:52.091623 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:52.092362 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:52.093862 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:52.094774 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:52.095479 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:52.095641 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:52.098536 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:52.098542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:52.098555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:52.098583 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:52.100405 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:52.100935 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:52.100945 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:52.100949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:52.101059 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:52.101447 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:52.101455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:52.101456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:52.101835 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:52.102053 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:52.102389 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:12:52.102612 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":117,"threads_started":1}
I20260811 05:12:52.105131 23650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } }
I20260811 05:12:52.105406 23650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:52.105476 23650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0903a134d5a7443984a9827f3106a874, State: Initialized, Role: FOLLOWER
I20260811 05:12:52.106033 23650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } }
I20260811 05:12:52.106220 23650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:52.106248 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:52.106324 23650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:52.106462 23650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:52.107539 23650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } }
I20260811 05:12:52.107923 23650 leader_election.cc:302] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0903a134d5a7443984a9827f3106a874; no voters:
I20260811 05:12:52.108295 23650 leader_election.cc:288] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:52.108392 23659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:52.108686 23659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [term 1 LEADER]: Becoming Leader. State: Replica: 0903a134d5a7443984a9827f3106a874, State: Running, Role: LEADER
I20260811 05:12:52.109103 23659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } }
I20260811 05:12:52.109591 23650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:52.110755 23662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0903a134d5a7443984a9827f3106a874. Latest consensus state: current_term: 1 leader_uuid: "0903a134d5a7443984a9827f3106a874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } } }
I20260811 05:12:52.111012 23662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:52.111414 23661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0903a134d5a7443984a9827f3106a874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0903a134d5a7443984a9827f3106a874" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 33779 } } }
I20260811 05:12:52.111598 23661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:52.111960 23670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:52.115378 23670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:52.118304 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:52.108426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45483 (local address 127.30.88.254:33779)
0811 05:12:52.108550 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:52.108554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:52.110371 (+ 1817us) server_negotiation.cc:407] Connection header received
0811 05:12:52.110564 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:52.110567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:52.110611 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:52.110664 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:52.111648 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:52.112418 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:52.113254 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:52.113506 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:52.113612 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:52.115053 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:52.115057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:52.115060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:52.115084 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:52.116551 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:52.117031 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:52.117035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:52.117037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:52.117118 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:52.117450 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:52.117453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:52.117455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:52.117629 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:52.117783 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:52.118051 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 05:12:52.118145 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:12:52.121543 23670 catalog_manager.cc:1347] Generated new cluster ID: 2b3d704cf869424d85885e591e453155
I20260811 05:12:52.121618 23670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:52.129967 23670 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:52.130793 23670 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:52.139325 23670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0903a134d5a7443984a9827f3106a874: Generated new TSK 0
I20260811 05:12:52.140128 23670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:52.149500 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23570
W20260811 05:12:52.158288 23551 connection.cc:535] client connection to 127.30.88.254:33779 recv error: Network error: failed to read from TLS socket (remote: 127.30.88.254:33779): Connection reset by peer (error 104)
2026-08-11T05:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:52 dist-test-slave-7c35 krb5kdc[23682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:52 dist-test-slave-7c35 krb5kdc[23682](info): set up 2 sockets
Aug 11 05:12:52 dist-test-slave-7c35 krb5kdc[23682](info): commencing operation
krb5kdc: starting...
W20260811 05:12:53.260810 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:53 dist-test-slave-7c35 krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:53.385036 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43975
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:53.493541 23696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:53.493839 23696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:53.493920 23696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:53.497809 23696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:53.497885 23696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:53.497921 23696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:53.497972 23696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:53.498004 23696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:53.502977 23696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:43975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:53.503907 23696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:53.504877 23696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:53.513157 23703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:53.513177 23706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:53.513228 23696 server_base.cc:1034] running on GCE node
W20260811 05:12:53.513177 23702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:53.514014 23696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:53.515091 23696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:53.516278 23696 hybrid_clock.cc:648] HybridClock initialized: now 1786425173516245 us; error 44 us; skew 500 ppm
Aug 11 05:12:53 dist-test-slave-7c35 krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:53.519127 23696 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:53.520118 23696 webserver.cc:460] Webserver started at http://127.30.88.254:35229/ using document root <none> and password file <none>
I20260811 05:12:53.520718 23696 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:53.520810 23696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:53.521052 23696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:53.523171 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "0e73fd24ff554bbd9e4ee20c3c7da338"
format_stamp: "Formatted at 2026-08-11 05:12:53 on dist-test-slave-7c35"
server_key: "9e74ad2ca9381cf5f8009fdfe234d4cc"
server_key_iv: "78a9691fa918553168fab030638dd8c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:53.523733 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "0e73fd24ff554bbd9e4ee20c3c7da338"
format_stamp: "Formatted at 2026-08-11 05:12:53 on dist-test-slave-7c35"
server_key: "9e74ad2ca9381cf5f8009fdfe234d4cc"
server_key_iv: "78a9691fa918553168fab030638dd8c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:53.527360 23696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:53.529732 23713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:53.531040 23696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:53.531178 23696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "0e73fd24ff554bbd9e4ee20c3c7da338"
format_stamp: "Formatted at 2026-08-11 05:12:53 on dist-test-slave-7c35"
server_key: "9e74ad2ca9381cf5f8009fdfe234d4cc"
server_key_iv: "78a9691fa918553168fab030638dd8c8"
server_key_version: "encryptionkey@0"
I20260811 05:12:53.531288 23696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:53.544303 23696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:53.547569 23696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:53.547890 23696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:53.555227 23696 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:43975
I20260811 05:12:53.555253 23771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:43975 every 8 connection(s)
I20260811 05:12:53.556162 23696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:53.558782 23773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:53.561682 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23696
I20260811 05:12:53.561839 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:53.562148 31075 external_mini_cluster.cc:1428] Setting key b45e8706831236dfd22ab5f5c81efee6
I20260811 05:12:53.564864 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338: Bootstrap starting.
I20260811 05:12:53.567065 23773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:53.567806 23773 log.cc:824] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:53.569608 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338: No bootstrap required, opened a new log
I20260811 05:12:53.571238 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:53.563602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60283 (local address 127.30.88.254:43975)
0811 05:12:53.564034 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:53.564048 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:53.564083 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:53.564708 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:53.564726 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:53.565113 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:53.565435 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:53.565845 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:53.566846 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:53.567861 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:53.568381 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:53.568558 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:53.569488 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:53.569493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:53.569506 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:53.569917 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:53.569981 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:53.570005 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:53.570162 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:53.570350 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 05:12:53.570487 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":136,"threads_started":1}
I20260811 05:12:53.573688 23773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } }
I20260811 05:12:53.573946 23773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:53.574020 23773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e73fd24ff554bbd9e4ee20c3c7da338, State: Initialized, Role: FOLLOWER
I20260811 05:12:53.574226 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:53.574476 23773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } }
I20260811 05:12:53.574617 23773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:53.574704 23773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:53.574820 23773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:53.575794 23773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } }
I20260811 05:12:53.576124 23773 leader_election.cc:302] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e73fd24ff554bbd9e4ee20c3c7da338; no voters:
I20260811 05:12:53.576447 23773 leader_election.cc:288] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:53.576632 23784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:53.576872 23784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [term 1 LEADER]: Becoming Leader. State: Replica: 0e73fd24ff554bbd9e4ee20c3c7da338, State: Running, Role: LEADER
I20260811 05:12:53.577222 23784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } }
I20260811 05:12:53.577780 23773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:53.578339 23784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e73fd24ff554bbd9e4ee20c3c7da338. Latest consensus state: current_term: 1 leader_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } } }
I20260811 05:12:53.578521 23784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:53.578827 23786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73fd24ff554bbd9e4ee20c3c7da338" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 43975 } } }
I20260811 05:12:53.578972 23786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:53.579068 23795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:53.581671 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:53.576409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52283 (local address 127.30.88.254:43975)
0811 05:12:53.576535 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:53.576539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:53.576838 (+ 299us) server_negotiation.cc:407] Connection header received
0811 05:12:53.577053 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:53.577056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:53.577099 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:53.577146 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:53.577535 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:53.578261 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:53.579432 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:53.579752 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:53.579855 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:53.581018 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:53.581022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:53.581025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:53.581038 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:53.581064 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:53.581077 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:53.581135 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:53.581515 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 05:12:53.581551 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:12:53.582653 23795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:53.588658 23795 catalog_manager.cc:1347] Generated new cluster ID: f368ab864d9247338b11a0904bb8772c
I20260811 05:12:53.588737 23795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:53.604796 23795 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:53.606021 23795 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:53.618090 23795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e73fd24ff554bbd9e4ee20c3c7da338: Generated new TSK 0
I20260811 05:12:53.619143 23795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:53.630339 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23696
W20260811 05:12:53.640453 23680 connection.cc:535] client connection to 127.30.88.254:43975 recv error: Network error: failed to read from TLS socket (remote: 127.30.88.254:43975): Connection reset by peer (error 104)
2026-08-11T05:12:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:53 dist-test-slave-7c35 krb5kdc[23809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:53 dist-test-slave-7c35 krb5kdc[23809](info): set up 2 sockets
Aug 11 05:12:53 dist-test-slave-7c35 krb5kdc[23809](info): commencing operation
krb5kdc: starting...
W20260811 05:12:54.737399 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:54 dist-test-slave-7c35 krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:54.859972 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:42317
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:42317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:54.969529 23823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:54.969769 23823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:54.969832 23823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:54.973829 23823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:54.973886 23823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:54.973905 23823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:54.973922 23823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:54.973959 23823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:54.979131 23823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:42317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:42317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:54.980109 23823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:54.981144 23823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:54.989331 23832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.989344 23829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.989344 23828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:54.989588 23823 server_base.cc:1034] running on GCE node
I20260811 05:12:54.990289 23823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:54.991395 23823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:54.992578 23823 hybrid_clock.cc:648] HybridClock initialized: now 1786425174992549 us; error 43 us; skew 500 ppm
Aug 11 05:12:54 dist-test-slave-7c35 krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:54.999938 23823 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:55.000895 23823 webserver.cc:460] Webserver started at http://127.30.88.254:33577/ using document root <none> and password file <none>
I20260811 05:12:55.001462 23823 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:55.001556 23823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:55.001789 23823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:55.003928 23823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "5fa3fc7c2666453ab33b2303b9480eb9"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-7c35"
server_key: "f5a3d4732f5e0acd601ae040f765993b"
server_key_iv: "0caafa4f675bb28b417e5b03d7ae5ac5"
server_key_version: "encryptionkey@0"
I20260811 05:12:55.004503 23823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "5fa3fc7c2666453ab33b2303b9480eb9"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-7c35"
server_key: "f5a3d4732f5e0acd601ae040f765993b"
server_key_iv: "0caafa4f675bb28b417e5b03d7ae5ac5"
server_key_version: "encryptionkey@0"
I20260811 05:12:55.008163 23823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:12:55.010519 23839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:55.011708 23823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:12:55.011848 23823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "5fa3fc7c2666453ab33b2303b9480eb9"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-7c35"
server_key: "f5a3d4732f5e0acd601ae040f765993b"
server_key_iv: "0caafa4f675bb28b417e5b03d7ae5ac5"
server_key_version: "encryptionkey@0"
I20260811 05:12:55.011957 23823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:55.016438 23823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:55.019213 23823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:55.019395 23823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:55.026636 23823 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:42317
I20260811 05:12:55.026651 23897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:42317 every 8 connection(s)
I20260811 05:12:55.027592 23823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:55.030705 23898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:55.035880 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23823
I20260811 05:12:55.035928 23898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9: Bootstrap starting.
I20260811 05:12:55.036057 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:55.036327 31075 external_mini_cluster.cc:1428] Setting key df89fe59057420e74a30ca6add4fb311
I20260811 05:12:55.038401 23898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:55.039232 23898 log.cc:824] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:55.041167 23898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9: No bootstrap required, opened a new log
I20260811 05:12:55.042712 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:55.037579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43051 (local address 127.30.88.254:42317)
0811 05:12:55.037968 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:55.037982 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:55.038018 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:12:55.038660 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:55.038679 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:55.039041 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:55.039316 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:55.041360 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:55.041367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:55.041382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:55.041395 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:55.041459 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:55.041501 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:55.041609 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:55.041741 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 05:12:55.041880 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
I20260811 05:12:55.045354 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:55.045485 23898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } }
I20260811 05:12:55.045698 23898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:55.045768 23898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa3fc7c2666453ab33b2303b9480eb9, State: Initialized, Role: FOLLOWER
I20260811 05:12:55.046208 23898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } }
I20260811 05:12:55.046325 23898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:55.046401 23898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:55.046550 23898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:55.047565 23898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } }
I20260811 05:12:55.047991 23898 leader_election.cc:302] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fa3fc7c2666453ab33b2303b9480eb9; no voters:
I20260811 05:12:55.048398 23898 leader_election.cc:288] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:55.048489 23910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:55.048715 23910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [term 1 LEADER]: Becoming Leader. State: Replica: 5fa3fc7c2666453ab33b2303b9480eb9, State: Running, Role: LEADER
I20260811 05:12:55.049054 23910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } }
I20260811 05:12:55.049357 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:55.047584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48691 (local address 127.30.88.254:42317)
0811 05:12:55.047696 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:55.047700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:55.047850 (+ 150us) server_negotiation.cc:407] Connection header received
0811 05:12:55.047970 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:55.047972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:55.048000 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:55.048059 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:55.048836 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:55.048840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:55.048843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:55.048854 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:55.048878 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:55.048891 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:55.048944 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:55.049172 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 05:12:55.049225 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:12:55.051704 23898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:55.053476 23913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } } }
I20260811 05:12:55.053612 23913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:55.053890 23911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa3fc7c2666453ab33b2303b9480eb9. Latest consensus state: current_term: 1 leader_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fa3fc7c2666453ab33b2303b9480eb9" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 42317 } } }
I20260811 05:12:55.053970 23928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:55.054395 23911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:55.057679 23928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:55.062541 23928 catalog_manager.cc:1347] Generated new cluster ID: 038bdfd3130e4f4d96360b2abcd03323
I20260811 05:12:55.062608 23928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:55.075044 23928 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:55.075415 23928 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:55.082651 23928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fa3fc7c2666453ab33b2303b9480eb9: Generated new TSK 0
I20260811 05:12:55.083477 23928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:55.090754 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23823
2026-08-11T05:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:55 dist-test-slave-7c35 krb5kdc[23935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:55 dist-test-slave-7c35 krb5kdc[23935](info): set up 2 sockets
Aug 11 05:12:55 dist-test-slave-7c35 krb5kdc[23935](info): commencing operation
krb5kdc: starting...
W20260811 05:12:56.207590 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:56.334173 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41011
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:40953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:41011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:56.445505 23951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:56.445750 23951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:56.445791 23951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:56.449782 23951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:56.449841 23951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:56.449859 23951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:56.449875 23951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:56.449930 23951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:56.455271 23951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:40953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:41011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:41011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:56.456549 23951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:56.457599 23951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:56.465710 23956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:56.465742 23957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:56.465803 23951 server_base.cc:1034] running on GCE node
W20260811 05:12:56.465695 23959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:56.466535 23951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:56.467631 23951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:56.468807 23951 hybrid_clock.cc:648] HybridClock initialized: now 1786425176468774 us; error 44 us; skew 500 ppm
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:56.471845 23951 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:56.472756 23951 webserver.cc:460] Webserver started at http://127.30.88.254:36739/ using document root <none> and password file <none>
I20260811 05:12:56.473316 23951 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:56.473408 23951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:56.473650 23951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:56.475749 23951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "a240612562c6482fab31383b4f26e3b5"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-7c35"
server_key: "9ab01fd10837bd68185e4fac91e42856"
server_key_iv: "410c3afc89979ae2659385535700d447"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.476346 23951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "a240612562c6482fab31383b4f26e3b5"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-7c35"
server_key: "9ab01fd10837bd68185e4fac91e42856"
server_key_iv: "410c3afc89979ae2659385535700d447"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.479961 23951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 05:12:56.482223 23966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:56.483424 23951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:56.483613 23951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "a240612562c6482fab31383b4f26e3b5"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-7c35"
server_key: "9ab01fd10837bd68185e4fac91e42856"
server_key_iv: "410c3afc89979ae2659385535700d447"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.483722 23951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:56.494365 23951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:56.497118 23951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:56.497323 23951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:56.504441 23951 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:41011
I20260811 05:12:56.504472 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:41011 every 8 connection(s)
I20260811 05:12:56.505381 23951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:56.508020 24027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:56.510588 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 23951
I20260811 05:12:56.510749 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:56.511085 31075 external_mini_cluster.cc:1428] Setting key b09a35fb221d974232746586bbce027c
I20260811 05:12:56.513904 24027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5: Bootstrap starting.
I20260811 05:12:56.516033 24027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:56.516750 24027 log.cc:824] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:56.518679 24027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5: No bootstrap required, opened a new log
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:12:56.522051 24027 raft_consensus.cc:348] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } }
I20260811 05:12:56.522316 24027 raft_consensus.cc:374] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:56.522404 24027 raft_consensus.cc:729] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a240612562c6482fab31383b4f26e3b5, State: Initialized, Role: FOLLOWER
I20260811 05:12:56.522926 24027 consensus_queue.cc:260] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } }
I20260811 05:12:56.523063 24027 raft_consensus.cc:388] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:56.523146 24027 raft_consensus.cc:482] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:56.523244 24027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:56.524080 24027 raft_consensus.cc:504] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } }
I20260811 05:12:56.524463 24027 leader_election.cc:302] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a240612562c6482fab31383b4f26e3b5; no voters:
I20260811 05:12:56.524813 24027 leader_election.cc:288] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:56.524925 24032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:56.525161 24032 raft_consensus.cc:686] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [term 1 LEADER]: Becoming Leader. State: Replica: a240612562c6482fab31383b4f26e3b5, State: Running, Role: LEADER
I20260811 05:12:56.525486 24032 consensus_queue.cc:237] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } }
I20260811 05:12:56.526086 24027 sys_catalog.cc:564] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:56.526356 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:56.512507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33861 (local address 127.30.88.254:41011)
0811 05:12:56.512925 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:56.512940 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:56.512975 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:56.513661 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:56.513680 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:56.514012 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:56.514373 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:56.515265 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.516314 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.517233 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.517698 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.517880 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:56.520734 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:56.520739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:56.520752 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:56.520778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:56.523088 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.523613 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.523619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.523625 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.523749 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.524054 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.524057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.524059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.524516 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:56.524725 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:56.525049 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 05:12:56.525536 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260811 05:12:56.526681 24032 sys_catalog.cc:455] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a240612562c6482fab31383b4f26e3b5. Latest consensus state: current_term: 1 leader_uuid: "a240612562c6482fab31383b4f26e3b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } } }
I20260811 05:12:56.526903 24032 sys_catalog.cc:458] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:56.527170 24033 sys_catalog.cc:455] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a240612562c6482fab31383b4f26e3b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240612562c6482fab31383b4f26e3b5" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 41011 } } }
I20260811 05:12:56.527277 24033 sys_catalog.cc:458] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:56.527444 24040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:56.529614 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:56.530710 24040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:56.537132 24040 catalog_manager.cc:1347] Generated new cluster ID: 7d324a93c86047ff9a2bcaef290598b0
I20260811 05:12:56.537206 24040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:56.540596 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:56.531873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46819 (local address 127.30.88.254:41011)
0811 05:12:56.531966 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:56.531971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:56.532492 (+ 521us) server_negotiation.cc:407] Connection header received
0811 05:12:56.532608 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:56.532611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:56.532659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:56.532745 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:56.533406 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.533951 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.534867 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.535289 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.535412 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:56.536933 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:56.536936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:56.536939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:56.536966 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:56.538796 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.539314 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.539318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.539320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.539429 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.539737 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.539740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.539742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.539958 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:56.540087 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:56.540312 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 05:12:56.540444 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:12:56.543936 24040 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:56.544816 24040 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:56.549837 24040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a240612562c6482fab31383b4f26e3b5: Generated new TSK 0
I20260811 05:12:56.550419 24040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:56.554605 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 23951
2026-08-11T05:12:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[24060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[24060](info): set up 2 sockets
Aug 11 05:12:56 dist-test-slave-7c35 krb5kdc[24060](info): commencing operation
krb5kdc: starting...
W20260811 05:12:57.678918 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:57 dist-test-slave-7c35 krb5kdc[24060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:57.802234 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:36237
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:57.911908 24074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:57.912151 24074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:57.912190 24074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:57.916011 24074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:57.916057 24074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:57.916081 24074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:57.916098 24074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:57.916112 24074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:57.921167 24074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:57.922046 24074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:57.922967 24074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:12:57.931209 24074 server_base.cc:1034] running on GCE node
W20260811 05:12:57.931123 24080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:57.931133 24081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:57.931126 24084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:57.931963 24074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:57.933012 24074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:57.934202 24074 hybrid_clock.cc:648] HybridClock initialized: now 1786425177934179 us; error 45 us; skew 500 ppm
Aug 11 05:12:57 dist-test-slave-7c35 krb5kdc[24060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:57.936858 24074 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:57.937671 24074 webserver.cc:460] Webserver started at http://127.30.88.254:43623/ using document root <none> and password file <none>
I20260811 05:12:57.938225 24074 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:57.938315 24074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:57.938545 24074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:57.940573 24074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "808224b04094496ca01d8e1ab7f11976"
format_stamp: "Formatted at 2026-08-11 05:12:57 on dist-test-slave-7c35"
server_key: "de56fb729f419aae6f63da1b0d4a4ca9"
server_key_iv: "b9fa5322870d6a4f7b42281f33152b31"
server_key_version: "encryptionkey@0"
I20260811 05:12:57.941133 24074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "808224b04094496ca01d8e1ab7f11976"
format_stamp: "Formatted at 2026-08-11 05:12:57 on dist-test-slave-7c35"
server_key: "de56fb729f419aae6f63da1b0d4a4ca9"
server_key_iv: "b9fa5322870d6a4f7b42281f33152b31"
server_key_version: "encryptionkey@0"
I20260811 05:12:57.944722 24074 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260811 05:12:57.947075 24092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:57.948184 24074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:57.948323 24074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "808224b04094496ca01d8e1ab7f11976"
format_stamp: "Formatted at 2026-08-11 05:12:57 on dist-test-slave-7c35"
server_key: "de56fb729f419aae6f63da1b0d4a4ca9"
server_key_iv: "b9fa5322870d6a4f7b42281f33152b31"
server_key_version: "encryptionkey@0"
I20260811 05:12:57.948433 24074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:57.964859 24074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:57.967752 24074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:57.967958 24074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:57.975070 24074 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:36237
I20260811 05:12:57.975133 24153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:36237 every 8 connection(s)
I20260811 05:12:57.976022 24074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:57.978648 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24074
I20260811 05:12:57.978791 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:57.979017 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:57.979149 31075 external_mini_cluster.cc:1428] Setting key f47cd158b56bb0844549f03127606683
I20260811 05:12:57.985419 24154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976: Bootstrap starting.
I20260811 05:12:57.987883 24154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:57.988276 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:57.980523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37163 (local address 127.30.88.254:36237)
0811 05:12:57.981012 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:57.981027 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:57.981075 (+ 48us) server_negotiation.cc:407] Connection header received
0811 05:12:57.981804 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:57.981822 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:57.982149 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:57.982470 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:57.982959 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:57.984002 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:57.984919 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:57.985391 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:57.985565 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:57.986458 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:57.986463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:57.986474 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:57.986894 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:57.986966 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:57.986990 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:57.987103 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:57.987256 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:12:57.987459 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":146,"threads_started":1}
I20260811 05:12:57.988672 24154 log.cc:824] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:57.990769 24154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976: No bootstrap required, opened a new log
I20260811 05:12:57.991024 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:57.994730 24154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } }
I20260811 05:12:57.995043 24154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:57.995107 24154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808224b04094496ca01d8e1ab7f11976, State: Initialized, Role: FOLLOWER
I20260811 05:12:57.995507 24154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } }
I20260811 05:12:57.995695 24154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:57.995761 24154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:57.995860 24154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:57.996858 24154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } }
I20260811 05:12:57.997213 24154 leader_election.cc:302] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 808224b04094496ca01d8e1ab7f11976; no voters:
I20260811 05:12:57.997532 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:57.993025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33177 (local address 127.30.88.254:36237)
0811 05:12:57.993156 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:57.993162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:57.993383 (+ 221us) server_negotiation.cc:407] Connection header received
0811 05:12:57.993475 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:57.993478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:57.993527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:57.993623 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:57.993868 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:57.994616 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:57.995645 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:57.996130 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:57.996242 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:57.997126 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:57.997129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:57.997132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:57.997145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:57.997172 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:57.997184 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:57.997271 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:57.997338 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:12:57.997387 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:12:57.997557 24154 leader_election.cc:288] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:57.997651 24165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:57.997867 24165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [term 1 LEADER]: Becoming Leader. State: Replica: 808224b04094496ca01d8e1ab7f11976, State: Running, Role: LEADER
I20260811 05:12:57.998165 24165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } }
I20260811 05:12:57.998781 24154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:57.999373 24165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 808224b04094496ca01d8e1ab7f11976. Latest consensus state: current_term: 1 leader_uuid: "808224b04094496ca01d8e1ab7f11976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } } }
I20260811 05:12:57.999497 24165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:57.999886 24172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:57.999862 24166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "808224b04094496ca01d8e1ab7f11976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808224b04094496ca01d8e1ab7f11976" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36237 } } }
I20260811 05:12:57.999959 24166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:58.002632 24172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:58.008158 24172 catalog_manager.cc:1347] Generated new cluster ID: 5d84250ceb5244c0b4874ad22a7bda1d
I20260811 05:12:58.008239 24172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:58.020679 24172 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:58.021576 24172 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:58.026381 24172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 808224b04094496ca01d8e1ab7f11976: Generated new TSK 0
I20260811 05:12:58.027009 24172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:58.033479 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24074
2026-08-11T05:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:58 dist-test-slave-7c35 krb5kdc[24186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:58 dist-test-slave-7c35 krb5kdc[24186](info): set up 2 sockets
Aug 11 05:12:58 dist-test-slave-7c35 krb5kdc[24186](info): commencing operation
krb5kdc: starting...
W20260811 05:12:59.154187 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:12:59 dist-test-slave-7c35 krb5kdc[24186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:12:59.283973 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35503
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:38823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:59.404129 24200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:59.404413 24200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:59.404457 24200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:59.408495 24200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:59.408576 24200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:59.408598 24200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:59.408615 24200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:59.408631 24200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:59.414249 24200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:38823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:12:59.415433 24200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:59.416469 24200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:59.425128 24205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:59.425138 24206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:59.425156 24209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:59.425422 24200 server_base.cc:1034] running on GCE node
I20260811 05:12:59.426025 24200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:59.427278 24200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:59.428467 24200 hybrid_clock.cc:648] HybridClock initialized: now 1786425179428446 us; error 40 us; skew 500 ppm
Aug 11 05:12:59 dist-test-slave-7c35 krb5kdc[24186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425179, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:59.431394 24200 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:12:59.432367 24200 webserver.cc:460] Webserver started at http://127.30.88.254:34331/ using document root <none> and password file <none>
I20260811 05:12:59.432961 24200 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:59.433058 24200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:59.433324 24200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:59.435539 24200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "81df0ed0aeb440e48700cdc540815588"
format_stamp: "Formatted at 2026-08-11 05:12:59 on dist-test-slave-7c35"
server_key: "68c3891a834cd8ed080e9b54733f1f73"
server_key_iv: "87cca7e68a708cd5e8e50e50340c9d3a"
server_key_version: "encryptionkey@0"
I20260811 05:12:59.436168 24200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "81df0ed0aeb440e48700cdc540815588"
format_stamp: "Formatted at 2026-08-11 05:12:59 on dist-test-slave-7c35"
server_key: "68c3891a834cd8ed080e9b54733f1f73"
server_key_iv: "87cca7e68a708cd5e8e50e50340c9d3a"
server_key_version: "encryptionkey@0"
I20260811 05:12:59.440026 24200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 05:12:59.442538 24217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:59.443817 24200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:12:59.443967 24200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "81df0ed0aeb440e48700cdc540815588"
format_stamp: "Formatted at 2026-08-11 05:12:59 on dist-test-slave-7c35"
server_key: "68c3891a834cd8ed080e9b54733f1f73"
server_key_iv: "87cca7e68a708cd5e8e50e50340c9d3a"
server_key_version: "encryptionkey@0"
I20260811 05:12:59.444098 24200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:59.448897 24200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:59.452157 24200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:59.452368 24200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:59.459625 24200 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:35503
I20260811 05:12:59.459632 24277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:35503 every 8 connection(s)
I20260811 05:12:59.460582 24200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:59.462023 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24200
I20260811 05:12:59.462177 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:12:59.462502 31075 external_mini_cluster.cc:1428] Setting key 42e9a330a966f2c72224b17e59153559
I20260811 05:12:59.463810 24279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:59.467769 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:59.463983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57225 (local address 127.30.88.254:35503)
0811 05:12:59.464418 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:59.464433 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:59.464468 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:12:59.465106 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:59.465122 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:59.465390 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:59.465733 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:59.466558 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:59.466573 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:59.466584 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:59.466595 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:59.466652 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:59.466672 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:59.466826 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:59.466916 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:12:59.467079 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":142,"threads_started":1}
I20260811 05:12:59.469892 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588: Bootstrap starting.
I20260811 05:12:59.470525 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:59.472311 24279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:59.473233 24279 log.cc:824] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:59.474519 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:59.472691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50485 (local address 127.30.88.254:35503)
0811 05:12:59.472823 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:59.472827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:59.473045 (+ 218us) server_negotiation.cc:407] Connection header received
0811 05:12:59.473174 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:59.473176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:59.473206 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:59.473269 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:59.474077 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:59.474080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:59.474085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:59.474100 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:59.474126 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:59.474139 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:59.474219 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:59.474340 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:12:59.474372 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:12:59.475577 24279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588: No bootstrap required, opened a new log
I20260811 05:12:59.479065 24279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } }
I20260811 05:12:59.479295 24279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:59.479374 24279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81df0ed0aeb440e48700cdc540815588, State: Initialized, Role: FOLLOWER
I20260811 05:12:59.479820 24279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } }
I20260811 05:12:59.480002 24279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:59.480078 24279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:59.480154 24279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:59.481051 24279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } }
I20260811 05:12:59.481336 24279 leader_election.cc:302] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81df0ed0aeb440e48700cdc540815588; no voters:
I20260811 05:12:59.481591 24279 leader_election.cc:288] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:59.481714 24290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:59.481966 24290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [term 1 LEADER]: Becoming Leader. State: Replica: 81df0ed0aeb440e48700cdc540815588, State: Running, Role: LEADER
I20260811 05:12:59.482334 24290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } }
I20260811 05:12:59.482889 24279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:59.484187 24292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81df0ed0aeb440e48700cdc540815588. Latest consensus state: current_term: 1 leader_uuid: "81df0ed0aeb440e48700cdc540815588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } } }
I20260811 05:12:59.484208 24291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81df0ed0aeb440e48700cdc540815588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df0ed0aeb440e48700cdc540815588" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 35503 } } }
I20260811 05:12:59.484366 24291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:59.484647 24292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:59.485388 24301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:59.488629 24301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:59.494496 24301 catalog_manager.cc:1347] Generated new cluster ID: b57fa7e39ec54e8d812b089140826776
I20260811 05:12:59.494568 24301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:59.506985 24301 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:59.507311 24301 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:59.512656 24301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81df0ed0aeb440e48700cdc540815588: Generated new TSK 0
I20260811 05:12:59.513267 24301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:59.522444 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24200
2026-08-11T05:12:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1488 ms)
[----------] 18 tests from AuthTokenIssuingTest (26372 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:59 dist-test-slave-7c35 krb5kdc[24312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:59 dist-test-slave-7c35 krb5kdc[24312](info): set up 2 sockets
Aug 11 05:12:59 dist-test-slave-7c35 krb5kdc[24312](info): commencing operation
krb5kdc: starting...
W20260811 05:13:00.630874 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:13:00 dist-test-slave-7c35 krb5kdc[24312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:13:00.753897 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:36133
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:00.863777 24329 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:00.864092 24329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:00.864153 24329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:00.864219 24329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:00.868150 24329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:00.868225 24329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:00.868263 24329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:00.868296 24329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:00.868330 24329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:00.873382 24329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:36133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:00.874366 24329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:00.875342 24329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:00.883584 24337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.883580 24335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.883783 24334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.883929 24329 server_base.cc:1034] running on GCE node
I20260811 05:13:00.884498 24329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:00.885591 24329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:00.886776 24329 hybrid_clock.cc:648] HybridClock initialized: now 1786425180886759 us; error 31 us; skew 500 ppm
Aug 11 05:13:00 dist-test-slave-7c35 krb5kdc[24312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:00.894560 24329 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:13:00.895610 24329 webserver.cc:460] Webserver started at http://127.30.88.254:42485/ using document root <none> and password file <none>
I20260811 05:13:00.896173 24329 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:00.896265 24329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:00.896502 24329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:00.898530 24329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "6cb6753f14e146ba9bfd8c07775e19c4"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-7c35"
server_key: "bf3a4658cd738b8d713315b632ac8087"
server_key_iv: "8b452d26fc2330988a4a594b2bd99391"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.899103 24329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "6cb6753f14e146ba9bfd8c07775e19c4"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-7c35"
server_key: "bf3a4658cd738b8d713315b632ac8087"
server_key_iv: "8b452d26fc2330988a4a594b2bd99391"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.902765 24329 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:13:00.905140 24344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.906369 24329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:00.906529 24329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "6cb6753f14e146ba9bfd8c07775e19c4"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-7c35"
server_key: "bf3a4658cd738b8d713315b632ac8087"
server_key_iv: "8b452d26fc2330988a4a594b2bd99391"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.906646 24329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:00.922658 24329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:00.925940 24329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:00.926146 24329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:00.933328 24329 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:36133
I20260811 05:13:00.933632 24400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:36133 every 8 connection(s)
I20260811 05:13:00.934278 24329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:00.937027 24401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:00.940415 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24329
I20260811 05:13:00.940577 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:13:00.940845 31075 external_mini_cluster.cc:1428] Setting key 95106c72e759a1a75b193f9c1886aaad
I20260811 05:13:00.940884 24401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.88.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.253@KRBTEST.COM" created.
Aug 11 05:13:00 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:13:00.959671 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.943204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:36133 (local address 127.0.0.1:38742)
0811 05:13:00.943653 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.943704 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:00.944834 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:00.945560 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:00.945569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:00.946097 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:00.946502 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.946525 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.947762 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.947768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.949121 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.949136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.949780 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.949787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.949926 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.950560 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:00.950579 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:00.954708 (+ 4129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:00.956772 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.956781 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.956790 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.957023 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.957344 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.957348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.957350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.957462 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.958026 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:00.958031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:00.958171 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 05:13:00.958332 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 05:13:00.958715 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":106,"thread_start_us":132,"threads_started":1}
I20260811 05:13:00.959672 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.943204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.30.88.254:36133)
0811 05:13:00.943711 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.943724 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.943803 (+ 79us) server_negotiation.cc:407] Connection header received
0811 05:13:00.945171 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.945176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.945343 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.945507 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.946669 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.947619 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.949253 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.949612 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.949729 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.954899 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.954905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.954915 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.954936 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.956641 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.957136 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.957141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.957143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.957249 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.957561 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.957564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.957565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.957909 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.958052 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.958415 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.958636 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"spinlock_wait_cycles":5888,"thread_start_us":140,"threads_started":1}
I20260811 05:13:00.962150 24401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:00.962924 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.962621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38038)
0811 05:13:00.962743 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.962808 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 05:13:00.963122 24347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
I20260811 05:13:00.963995 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.963746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38044)
0811 05:13:00.963862 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.963917 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 05:13:00.964145 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:41941: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
I20260811 05:13:00.969990 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:41941
--webserver_interface=127.30.88.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:01.013398 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } attempt: 1
I20260811 05:13:01.014335 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.013977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38048)
0811 05:13:01.014168 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.014241 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 05:13:01.015229 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.014992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38064)
0811 05:13:01.015106 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.015161 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 05:13:01.015405 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:41941: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
W20260811 05:13:01.082691 24409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:01.083021 24409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:01.083076 24409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:01.083160 24409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:13:01.083598 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } attempt: 2
I20260811 05:13:01.084443 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.084117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38068)
0811 05:13:01.084281 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.084350 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 05:13:01.085266 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.085037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38076)
0811 05:13:01.085151 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.085203 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:13:01.085456 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:41941: Network error: Client connection negotiation failed: client connection to 127.30.88.253:41941: connect: Connection refused (error 111)
W20260811 05:13:01.087698 24409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:01.087770 24409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:01.087841 24409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:01.087878 24409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:01.087908 24409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:01.093062 24409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.88.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:01.094125 24409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:01.095223 24409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:01.103945 24418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:01.103961 24414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:01.103976 24415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:01.104127 24409 server_base.cc:1034] running on GCE node
I20260811 05:13:01.104734 24409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:01.105773 24409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:01.106925 24409 hybrid_clock.cc:648] HybridClock initialized: now 1786425181106897 us; error 41 us; skew 500 ppm
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:01.109730 24409 init.cc:377] Logged in from keytab as kudu/127.30.88.253@KRBTEST.COM (short username kudu)
I20260811 05:13:01.110608 24409 webserver.cc:460] Webserver started at http://127.30.88.253:46661/ using document root <none> and password file <none>
I20260811 05:13:01.111213 24409 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:01.111304 24409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:01.111534 24409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:01.113499 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data/instance:
uuid: "0c780041dc70408e926ab881adb240d9"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "575433b905c4b6a04063d745c60fb584"
server_key_iv: "82da48a7c4f6aa6019e4e72e6c2924c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.114074 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal/instance:
uuid: "0c780041dc70408e926ab881adb240d9"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "575433b905c4b6a04063d745c60fb584"
server_key_iv: "82da48a7c4f6aa6019e4e72e6c2924c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.117619 24409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 05:13:01.119908 24425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:01.121042 24409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:01.121181 24409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "0c780041dc70408e926ab881adb240d9"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "575433b905c4b6a04063d745c60fb584"
server_key_iv: "82da48a7c4f6aa6019e4e72e6c2924c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.121287 24409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:01.160452 24409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:01.163861 24409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:01.164073 24409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:01.171164 24409 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.253:41941
I20260811 05:13:01.171480 24489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.253:41941 every 8 connection(s)
I20260811 05:13:01.172489 24409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:01.175429 24490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:01.176644 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } attempt: 3
I20260811 05:13:01.176762 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24409
I20260811 05:13:01.176882 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-1/wal/instance
I20260811 05:13:01.177256 31075 external_mini_cluster.cc:1428] Setting key 7d7e19932fee9c8a6a49fd6fec259fae
I20260811 05:13:01.180087 24490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.88.252@KRBTEST.COM; defaulting to no policy
I20260811 05:13:01.191339 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.177225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38086)
0811 05:13:01.177457 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.177473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.177575 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.180341 (+ 2766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.180344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.180563 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.180700 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.180708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.182154 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.182157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.182976 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.182989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.184070 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.184074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.184175 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.184690 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.184705 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.186432 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.189180 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.189184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.189186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.189571 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.189934 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.189937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.189939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.190032 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.190738 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.190741 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.190841 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.191017 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.191165 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"spinlock_wait_cycles":3328}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.252@KRBTEST.COM" created.
I20260811 05:13:01.192647 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.178145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.30.88.253:41941)
0811 05:13:01.178672 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.178693 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.178740 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:13:01.179714 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.179734 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.180067 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.180394 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.180827 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.181821 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.183089 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.183937 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.184122 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.186634 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.186644 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.186659 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.186696 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.189090 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.189706 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.189718 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.189722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.189820 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.190144 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.190148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.190150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.190621 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.190760 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.191176 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.191470 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":158,"threads_started":1}
I20260811 05:13:01.192981 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.181958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:36133 (local address 127.0.0.1:38750)
0811 05:13:01.182396 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.182461 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.182613 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.183006 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.183013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.183360 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.183487 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.183493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.184318 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.184322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.185279 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.185291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.186634 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.186640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.186753 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.187465 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.187477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.189024 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.191096 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.191105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.191110 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.191332 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.191772 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.191775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.191776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.191898 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.192348 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.192358 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.192479 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.192623 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.192801 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":150,"threads_started":1}
I20260811 05:13:01.192973 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.182022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.30.88.254:36133)
0811 05:13:01.182154 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.182158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.182531 (+ 373us) server_negotiation.cc:407] Connection header received
0811 05:13:01.182767 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.182773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.182835 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.182938 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.183621 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.184172 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.185416 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.186470 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.186570 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.189165 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.189170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.189173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.189198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.190972 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.191582 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.191585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.191587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.191668 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.191994 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.191996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.191998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.192202 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.192314 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.192691 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.192801 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":2304}
I20260811 05:13:01.194262 24490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.194958 24401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.195828 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.195423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43060)
0811 05:13:01.195561 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.195736 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 05:13:01.196588 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.196329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43062)
0811 05:13:01.196484 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.196533 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 05:13:01.196753 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:01.202916 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.195086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38102)
0811 05:13:01.195194 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.195206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.195300 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.195608 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.195611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.195779 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.195850 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.195855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.196763 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.196766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.197318 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.197326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.197808 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.197815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.197941 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.198367 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.198387 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.199706 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.201434 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.201437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.201439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.201624 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.201899 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.201902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.201904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.201984 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.202364 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.202367 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.202451 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.202580 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.202712 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 05:13:01.202986 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.195298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.30.88.253:41941)
0811 05:13:01.195422 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.195426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.195440 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:01.195477 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.195479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.195520 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.195598 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.195983 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.196651 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.197425 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.197675 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.197770 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.199835 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.199839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.199841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.199867 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.201292 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.201720 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.201724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.201726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.201799 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.202078 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.202081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.202082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.202238 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.202364 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.202614 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.202771 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":53}
I20260811 05:13:01.203666 24490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.204389 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.204066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43070)
0811 05:13:01.204196 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.204318 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:13:01.204968 24430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
I20260811 05:13:01.205957 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.205728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43080)
0811 05:13:01.205854 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.205904 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 05:13:01.206131 24490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
I20260811 05:13:01.210055 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:34821
--webserver_interface=127.30.88.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:01.227936 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 1
I20260811 05:13:01.228790 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.228484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43084)
0811 05:13:01.228646 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.228713 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 05:13:01.229547 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.229327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43100)
0811 05:13:01.229434 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.229484 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:13:01.229714 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
I20260811 05:13:01.255414 24490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 1
I20260811 05:13:01.256296 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.255994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43110)
0811 05:13:01.256142 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.256217 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 05:13:01.257113 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.256886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43114)
0811 05:13:01.257004 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.257058 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:13:01.257289 24490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
I20260811 05:13:01.304893 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 2
I20260811 05:13:01.305809 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.305510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43118)
0811 05:13:01.305668 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.305730 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 05:13:01.306581 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.306389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43122)
0811 05:13:01.306485 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.306527 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:13:01.306782 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
W20260811 05:13:01.322526 24498 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:01.322801 24498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:01.322892 24498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:01.322986 24498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:13:01.325444 24490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 2
I20260811 05:13:01.326175 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.325902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43138)
0811 05:13:01.326032 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.326087 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:13:01.326969 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.326740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43146)
0811 05:13:01.326830 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.326920 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 05:13:01.327154 24490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
W20260811 05:13:01.327699 24498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:01.327746 24498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:01.327766 24498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:01.327816 24498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:01.327840 24498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:01.332957 24498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:36133,127.30.88.253:41941,127.30.88.252:34821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:34821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.88.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:01.333957 24498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:01.335052 24498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:01.343420 24506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:01.343441 24508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:01.343587 24498 server_base.cc:1034] running on GCE node
W20260811 05:13:01.343716 24504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:01.344276 24498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:01.345292 24498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:01.346477 24498 hybrid_clock.cc:648] HybridClock initialized: now 1786425181346442 us; error 49 us; skew 500 ppm
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:01.349123 24498 init.cc:377] Logged in from keytab as kudu/127.30.88.252@KRBTEST.COM (short username kudu)
I20260811 05:13:01.349920 24498 webserver.cc:460] Webserver started at http://127.30.88.252:39989/ using document root <none> and password file <none>
I20260811 05:13:01.350471 24498 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:01.350562 24498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:01.350791 24498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:01.352757 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data/instance:
uuid: "6b3edd81fb854897b9970178ebdacfc5"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "938a95a53c81dc577aa83d5bbcd3735b"
server_key_iv: "23826ffc4a7e69ae2ed9f77089c61645"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.353322 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal/instance:
uuid: "6b3edd81fb854897b9970178ebdacfc5"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "938a95a53c81dc577aa83d5bbcd3735b"
server_key_iv: "23826ffc4a7e69ae2ed9f77089c61645"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.356860 24498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:13:01.359165 24516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:01.360319 24498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:01.360456 24498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "6b3edd81fb854897b9970178ebdacfc5"
format_stamp: "Formatted at 2026-08-11 05:13:01 on dist-test-slave-7c35"
server_key: "938a95a53c81dc577aa83d5bbcd3735b"
server_key_iv: "23826ffc4a7e69ae2ed9f77089c61645"
server_key_version: "encryptionkey@0"
I20260811 05:13:01.360567 24498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:01.397272 24498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:01.400319 24498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:01.400533 24498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:01.406073 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 3
I20260811 05:13:01.406978 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.406636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43154)
0811 05:13:01.406762 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.406822 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 05:13:01.407742 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.407458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43166)
0811 05:13:01.407573 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.407629 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:13:01.407887 24498 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.252:34821
W20260811 05:13:01.407950 24401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:34821: Network error: Client connection negotiation failed: client connection to 127.30.88.252:34821: connect: Connection refused (error 111)
I20260811 05:13:01.408725 24576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.252:34821 every 8 connection(s)
I20260811 05:13:01.409237 24498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:01.412360 24577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:01.416167 24577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.416889 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24498
I20260811 05:13:01.417037 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786424902138073-31075-0/minicluster-data/master-2/wal/instance
I20260811 05:13:01.417349 31075 external_mini_cluster.cc:1428] Setting key b9a0bf8f16abf67d5082177196f95971
I20260811 05:13:01.418323 24490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 3
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:13:01.431339 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.418268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.30.88.254:36133)
0811 05:13:01.418425 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.418431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.419390 (+ 959us) server_negotiation.cc:407] Connection header received
0811 05:13:01.420450 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.420453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.420499 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.420596 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.421936 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.422697 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.423750 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.424085 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.424214 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.426991 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.426994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.426996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.427018 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.428864 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.429456 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.429461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.429463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.429586 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.429979 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.429982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.429984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.430248 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.430357 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.431004 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.431134 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:01.432296 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.418701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:36133 (local address 127.0.0.1:38754)
0811 05:13:01.419227 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.419282 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.420220 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.420659 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.420668 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.421136 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.421782 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.421795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.422814 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.422824 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.423593 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.423602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.424370 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.424376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.424538 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.425225 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.425249 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.426775 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.428986 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.428996 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.429008 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.429315 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.429697 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.429702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.429706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.429867 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.430411 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.430417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.430683 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.430920 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.431299 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":46,"spinlock_wait_cycles":3328,"thread_start_us":164,"threads_started":1}
I20260811 05:13:01.432294 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.419572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.30.88.252:34821)
0811 05:13:01.419817 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.419827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.419841 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:01.420229 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.420244 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.420491 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.421347 (+ 856us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.421545 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.422677 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.423870 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.424250 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.424413 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.427975 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.427987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.427996 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.428016 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.429636 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.430174 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.430180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.430182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.430270 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.430643 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.430646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.430648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.430945 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.431783 (+ 838us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.431880 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.432010 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":181,"spinlock_wait_cycles":55424,"thread_start_us":77,"threads_started":1}
I20260811 05:13:01.433061 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.418771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43182)
0811 05:13:01.418921 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.418941 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.419045 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.420675 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.420678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.420901 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.421025 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.421032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.422664 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.422668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.423561 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.423571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.424755 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.424758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.424822 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.425345 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.425362 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.427753 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.431495 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.431502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.431504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.431752 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.432049 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.432053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.432055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.432142 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.432578 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.432583 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.432683 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.432760 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.432892 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 05:13:01.433678 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.418956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.30.88.252:34821)
0811 05:13:01.419376 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.419387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.419422 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:13:01.420220 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.420243 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.420479 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.421192 (+ 713us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.421597 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.422548 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.423687 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.424251 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.424409 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.428432 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.428442 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.428445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.428475 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.431357 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.431876 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.431878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.431879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.431946 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.432243 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.432246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.432248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.432440 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.433316 (+ 876us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.433415 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.433523 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":323,"spinlock_wait_cycles":38912,"thread_start_us":125,"threads_started":1}
I20260811 05:13:01.433730 24577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.436986 24490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: Bootstrap starting.
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:01.439663 24490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:01.440608 24490 log.cc:824] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:13:01.443097 24490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: No bootstrap required, opened a new log
I20260811 05:13:01.443408 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.434344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:41941 (local address 127.0.0.1:38116)
0811 05:13:01.434581 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.434597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.434679 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.435066 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.435069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.435246 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.435340 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.435347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.436364 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.436366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.436868 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.436874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.437564 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.437567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.437648 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.438161 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.438175 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.439737 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.441530 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.441537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.441541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.441791 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.442271 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.442274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.442275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.442370 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.442889 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.442892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.442982 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.443097 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.443240 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260811 05:13:01.443406 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.434460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.30.88.253:41941)
0811 05:13:01.434578 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.434582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.434641 (+ 59us) server_negotiation.cc:407] Connection header received
0811 05:13:01.434799 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.434802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.434879 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.434953 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.435519 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.436258 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.437021 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.437342 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.437413 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.439898 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.439905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.439909 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.439941 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.441409 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.441894 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.441897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.441899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.441987 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.442473 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.442477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.442479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.442758 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.442886 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.443150 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.443259 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:13:01.444533 24577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:01.445785 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.434962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.30.88.254:36133)
0811 05:13:01.435081 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.435085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.435150 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:13:01.435278 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.435281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.435319 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.435398 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.436142 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.436813 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.437889 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.438226 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.438333 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.441787 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.441791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.441794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.441818 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.443517 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.444052 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.444055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.444057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.444327 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.444671 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.444675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.444677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.444901 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.445134 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.445491 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.445614 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:01.447607 24490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.447868 24490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c780041dc70408e926ab881adb240d9, State: Initialized, Role: FOLLOWER
I20260811 05:13:01.448259 24490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.448836 24585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.449071 24585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:13:01.449656 24490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:01.454051 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.445077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43194)
0811 05:13:01.445309 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.445322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.445409 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.445699 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.445701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.445861 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.445955 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.445964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.446773 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.446775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.447582 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.447590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.448115 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.448118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.448222 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.448670 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.448685 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.449936 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.452203 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.452206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.452208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.452485 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.452770 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.452775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.452778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.452873 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.453469 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.453471 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.453580 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.453710 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.453857 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":22}
I20260811 05:13:01.454321 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.445284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.30.88.252:34821)
0811 05:13:01.445377 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.445382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.445394 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:01.445505 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.445507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.445542 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.445588 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.446068 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.446686 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.447686 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.448000 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.448118 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.450058 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.450062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.450064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.450097 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.452089 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.452585 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.452591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.452592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.452673 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.453041 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.453046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.453047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.453333 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.453410 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.453780 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.454173 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":38}
W20260811 05:13:01.456322 24600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:01.456453 24600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:01.457216 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: Bootstrap starting.
I20260811 05:13:01.455536 24409 master_runner.cc:186] Error getting master registration for 127.30.88.254:36133: OK, instance_id { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" instance_seqno: 1786425180931416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:13:01.458989 24409 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:13:01.459518 24577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:01.460363 24577 log.cc:824] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:01.460737 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.446998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.30.88.253:41941)
0811 05:13:01.447096 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.447100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.447176 (+ 76us) server_negotiation.cc:407] Connection header received
0811 05:13:01.447278 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.447281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.447320 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.447398 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.448140 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.448823 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.449808 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.450126 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.450231 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.453129 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.453132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.453135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.453156 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.455633 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.459388 (+ 3755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.459392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.459393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.459481 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.459822 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.459826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.459828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.460045 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.460236 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.460471 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.460591 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:13:01.462630 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: No bootstrap required, opened a new log
I20260811 05:13:01.464227 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:01.466382 24577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.466626 24577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3edd81fb854897b9970178ebdacfc5, State: Initialized, Role: FOLLOWER
I20260811 05:13:01.466987 24577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.465639 24373 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6cb6753f14e146ba9bfd8c07775e19c4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770
I20260811 05:13:01.467473 24603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.467648 24603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.468405 24577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:13:01.473325 24614 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:01.473459 24614 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:01.471751 24498 master_runner.cc:186] Error getting master registration for 127.30.88.254:36133: OK, instance_id { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" instance_seqno: 1786425180931416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:13:01.475723 24498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:13:01.572129 24401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } attempt: 4
Aug 11 05:13:01 dist-test-slave-7c35 krb5kdc[24312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:13:01.582682 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.572749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:34821 (local address 127.0.0.1:43202)
0811 05:13:01.572922 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:01.572942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:01.573058 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:01.573405 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:01.573408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:01.573636 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:01.573761 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.573768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.574808 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.574811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.575707 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:01.575719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.576430 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.576434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:01.576539 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.577164 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:01.577182 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:01.578908 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:01.581035 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.581041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.581044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.581304 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.581589 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:01.581593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:01.581595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:01.581697 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:01.582118 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:01.582122 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:01.582230 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 05:13:01.582343 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:13:01.582497 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 05:13:01.582695 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.573039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.30.88.252:34821)
0811 05:13:01.573156 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.573161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.573175 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:01.573225 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.573228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.573277 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.573371 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.573908 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.574701 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.575851 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.576182 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.576290 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.579064 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.579068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.579071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.579100 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.580905 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.581389 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.581393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.581395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.581486 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.581790 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.581794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.581796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.582010 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.582113 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.582412 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.582534 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:01.587126 24401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: Bootstrap starting.
I20260811 05:13:01.590144 24401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:01.591156 24401 log.cc:824] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:01.593729 24401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: No bootstrap required, opened a new log
I20260811 05:13:01.596994 24401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.597234 24401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb6753f14e146ba9bfd8c07775e19c4, State: Initialized, Role: FOLLOWER
I20260811 05:13:01.597605 24401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.598229 24617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.598524 24617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.599079 24401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:01.599134 24373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:13:01.599254 24373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:01.600536 24373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.602181 24458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0c780041dc70408e926ab881adb240d9"
I20260811 05:13:01.602295 24545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b3edd81fb854897b9970178ebdacfc5"
I20260811 05:13:01.602475 24458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:01.602564 24545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:01.603893 24458 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb6753f14e146ba9bfd8c07775e19c4 in term 1.
I20260811 05:13:01.603929 24545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb6753f14e146ba9bfd8c07775e19c4 in term 1.
I20260811 05:13:01.601531 24373 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0c780041dc70408e926ab881adb240d9 (127.30.88.253:41941), 6b3edd81fb854897b9970178ebdacfc5 (127.30.88.252:34821)
I20260811 05:13:01.604969 24346 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b3edd81fb854897b9970178ebdacfc5, 6cb6753f14e146ba9bfd8c07775e19c4; no voters:
W20260811 05:13:01.608454 24629 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:01.608551 24629 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:01.611872 24329 master_runner.cc:263] No leader master found from master 6cb6753f14e146ba9bfd8c07775e19c4
I20260811 05:13:01.612010 24329 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:13:01.612187 24617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:01.612598 24617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [term 1 LEADER]: Becoming Leader. State: Replica: 6cb6753f14e146ba9bfd8c07775e19c4, State: Running, Role: LEADER
I20260811 05:13:01.613018 24617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } }
I20260811 05:13:01.622802 24617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb6753f14e146ba9bfd8c07775e19c4. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.623100 24617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:01.631541 24641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:01.634346 24641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:01.639299 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.615252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.30.88.254:36133)
0811 05:13:01.615393 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.615399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.620427 (+ 5028us) server_negotiation.cc:407] Connection header received
0811 05:13:01.620770 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.620776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.620873 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.620937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.628686 (+ 7749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.629451 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.630690 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.631017 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.631093 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.634569 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.634573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.634576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.634606 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.636364 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.637662 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.637665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.637668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.637752 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.638032 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.638035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.638037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.638239 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.638351 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.639020 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.639137 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:01.643327 24545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [term 1 FOLLOWER]: Refusing update from remote peer 6cb6753f14e146ba9bfd8c07775e19c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:01.644202 24636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:01.644369 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.612950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.30.88.253:41941)
0811 05:13:01.613074 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.613079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.627143 (+ 14064us) server_negotiation.cc:407] Connection header received
0811 05:13:01.627332 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.627335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.627384 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.627437 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.628489 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.629309 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.630549 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.630906 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.630996 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.638690 (+ 7694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.638694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.638697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.638726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.640427 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.642995 (+ 2568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.642999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.643001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.643096 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.643400 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.643403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.643405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.643618 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.643730 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.644079 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.644203 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:01.648307 24603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb6753f14e146ba9bfd8c07775e19c4. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.648427 24603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.649346 24458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [term 1 FOLLOWER]: Refusing update from remote peer 6cb6753f14e146ba9bfd8c07775e19c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:01.649942 24636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:01.651880 24522 tablet.cc:2369] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:01.654042 24585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb6753f14e146ba9bfd8c07775e19c4. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.654158 24585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.655603 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.655715 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:01.657015 24603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.657126 24603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.657716 24643 mvcc.cc:205] Tried to move back new op lower bound from 7317197544002813952 to 7317197543922565120. Current Snapshot: MvccSnapshot[applied={T|T < 7317197544002813952}]
I20260811 05:13:01.658079 24642 mvcc.cc:205] Tried to move back new op lower bound from 7317197544002813952 to 7317197543922565120. Current Snapshot: MvccSnapshot[applied={T|T < 7317197544002813952}]
I20260811 05:13:01.658238 24617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.658344 24617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:01.659327 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:01.619652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.30.88.252:34821)
0811 05:13:01.619764 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.619768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.631929 (+ 12161us) server_negotiation.cc:407] Connection header received
0811 05:13:01.632120 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.632124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.632176 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.632230 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.636636 (+ 4406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.637440 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.645822 (+ 8382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:01.646206 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:01.646283 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:01.648040 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:01.648043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:01.648046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:01.648071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:01.649721 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.650957 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.650961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.650963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.651053 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:01.651362 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:01.651365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:01.651367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:01.651571 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:01.651680 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:01.659034 (+ 7354us) server_negotiation.cc:299] Negotiation successful
0811 05:13:01.659171 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:13:01.659421 24585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb6753f14e146ba9bfd8c07775e19c4" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 36133 } } peers { permanent_uuid: "0c780041dc70408e926ab881adb240d9" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 41941 } } peers { permanent_uuid: "6b3edd81fb854897b9970178ebdacfc5" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 34821 } } }
I20260811 05:13:01.659523 24585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:01.662006 24641 catalog_manager.cc:1347] Generated new cluster ID: b772920ed327489a954e6d28753eb7f0
I20260811 05:13:01.662073 24641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:01.685564 24641 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:01.686447 24641 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:01.702689 24641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb6753f14e146ba9bfd8c07775e19c4: Generated new TSK 0
I20260811 05:13:01.703588 24641 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:13:01.727312 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.725570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.30.88.253:41941)
0811 05:13:01.725759 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.725764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.726258 (+ 494us) server_negotiation.cc:407] Connection header received
0811 05:13:01.726415 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.726419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.726469 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.726521 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.727168 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38146: BlockingRecv error: recv got EOF from 127.0.0.1:38146 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260811 05:13:01.727434 24582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.725621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.30.88.252:34821)
0811 05:13:01.725847 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.725851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.726463 (+ 612us) server_negotiation.cc:407] Connection header received
0811 05:13:01.726634 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.726638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.726685 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.726734 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.727322 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43230: BlockingRecv error: recv got EOF from 127.0.0.1:43230 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
W20260811 05:13:01.727458 24406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.725760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.30.88.254:36133)
0811 05:13:01.725898 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.725902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.726156 (+ 254us) server_negotiation.cc:407] Connection header received
0811 05:13:01.726326 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.726332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.726415 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.726490 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.727331 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38784: BlockingRecv error: recv got EOF from 127.0.0.1:38784 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 05:13:01.730558 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.729637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.30.88.253:41941)
0811 05:13:01.729743 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.729747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.729863 (+ 116us) server_negotiation.cc:407] Connection header received
0811 05:13:01.729971 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.729974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.730043 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.730142 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.730464 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38156: BlockingRecv error: recv got EOF from 127.0.0.1:38156 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 05:13:01.734048 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.733284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.30.88.253:41941)
0811 05:13:01.733403 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.733406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.733416 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:01.733448 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.733450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.733485 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.733570 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.733946 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38168: BlockingRecv error: recv got EOF from 127.0.0.1:38168 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 05:13:01.738579 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.737551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.30.88.253:41941)
0811 05:13:01.737672 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.737678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.737885 (+ 207us) server_negotiation.cc:407] Connection header received
0811 05:13:01.737999 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.738002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.738049 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.738131 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.738495 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38176: BlockingRecv error: recv got EOF from 127.0.0.1:38176 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 05:13:01.745194 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.744190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.30.88.253:41941)
0811 05:13:01.744338 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.744343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.744458 (+ 115us) server_negotiation.cc:407] Connection header received
0811 05:13:01.744549 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.744552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.744606 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.744690 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.745109 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38190: BlockingRecv error: recv got EOF from 127.0.0.1:38190 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 05:13:01.756203 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.754989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.30.88.253:41941)
0811 05:13:01.755126 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.755132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.755309 (+ 177us) server_negotiation.cc:407] Connection header received
0811 05:13:01.755451 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.755454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.755496 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.755564 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.756085 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38192: BlockingRecv error: recv got EOF from 127.0.0.1:38192 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 05:13:01.775889 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.774537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.30.88.253:41941)
0811 05:13:01.774671 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.774676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.774835 (+ 159us) server_negotiation.cc:407] Connection header received
0811 05:13:01.775051 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.775055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.775131 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.775224 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.775759 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38196: BlockingRecv error: recv got EOF from 127.0.0.1:38196 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 05:13:01.811375 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.810096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.30.88.253:41941)
0811 05:13:01.810306 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.810310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.810463 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:13:01.810589 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.810591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.810640 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.810714 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.811244 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38210: BlockingRecv error: recv got EOF from 127.0.0.1:38210 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 05:13:01.879197 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:01.877978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.30.88.253:41941)
0811 05:13:01.878155 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:01.878159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:01.878212 (+ 53us) server_negotiation.cc:407] Connection header received
0811 05:13:01.878355 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:01.878358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:01.878411 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:01.878540 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:01.879052 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38226: BlockingRecv error: recv got EOF from 127.0.0.1:38226 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 05:13:02.010759 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:02.009365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.30.88.253:41941)
0811 05:13:02.009552 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.009556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.009714 (+ 158us) server_negotiation.cc:407] Connection header received
0811 05:13:02.009899 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.009902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.009955 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.010036 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:02.010627 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38232: BlockingRecv error: recv got EOF from 127.0.0.1:38232 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 05:13:02.270479 24492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:02.269312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.30.88.253:41941)
0811 05:13:02.269498 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.269503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.269520 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:13:02.269642 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.269645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.269704 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.269804 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:02.270346 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38238: BlockingRecv error: recv got EOF from 127.0.0.1:38238 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:02.457845 24600 catalog_manager.cc:1259] Loaded cluster ID: b772920ed327489a954e6d28753eb7f0
I20260811 05:13:02.457933 24600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: loading cluster ID for follower catalog manager: success
I20260811 05:13:02.459568 24600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: acquiring CA information for follower catalog manager: success
I20260811 05:13:02.460753 24600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c780041dc70408e926ab881adb240d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:02.474633 24614 catalog_manager.cc:1259] Loaded cluster ID: b772920ed327489a954e6d28753eb7f0
I20260811 05:13:02.474722 24614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: loading cluster ID for follower catalog manager: success
I20260811 05:13:02.476344 24614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: acquiring CA information for follower catalog manager: success
I20260811 05:13:02.477443 24614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b3edd81fb854897b9970178ebdacfc5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:02.788333 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.784593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.30.88.253:41941)
0811 05:13:02.784847 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.784853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.784870 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:13:02.785049 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.785051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.785174 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.785283 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:02.785549 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.786062 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.786892 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.787102 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.787206 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.787518 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:02.787934 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:02.788067 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.788121 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.788175 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":65,"threads_started":1}
I20260811 05:13:02.794373 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.791380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.30.88.252:34821)
0811 05:13:02.791615 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.791619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.791632 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:02.791708 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.791710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.791810 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.791903 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:02.792085 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.792597 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.793286 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.793489 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.793557 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.793777 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:02.794084 (+ 307us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:02.794159 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.794236 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.794277 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260811 05:13:02.801189 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.797387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.30.88.252:34821)
0811 05:13:02.797500 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.797506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.797519 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:02.797560 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.797563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.797603 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.797683 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:02.797962 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.798716 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.799903 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.800164 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.800278 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.800701 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:02.800774 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:02.800846 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.800975 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.801027 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:02.802029 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.797104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.30.88.253:41941)
0811 05:13:02.797295 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.797299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.798062 (+ 763us) server_negotiation.cc:407] Connection header received
0811 05:13:02.798189 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.798192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.798239 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.798320 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:02.798560 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.799273 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.800592 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.800992 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.801100 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.801581 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:02.801655 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:02.801736 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.801830 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.801880 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 05:13:02.802450 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.797083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.30.88.254:36133)
0811 05:13:02.797617 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.797622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.797637 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:02.797807 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.797810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.797974 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.798083 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:02.798436 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.799154 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.800615 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.801001 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.801096 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.801601 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:02.801917 (+ 316us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:02.802015 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.802293 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.802344 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":84,"threads_started":1}
I20260811 05:13:02.805541 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24329
I20260811 05:13:02.817219 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24409
I20260811 05:13:02.824859 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24498
2026-08-11T05:13:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3304 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:02 dist-test-slave-7c35 krb5kdc[24747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:02 dist-test-slave-7c35 krb5kdc[24747](info): set up 2 sockets
Aug 11 05:13:02 dist-test-slave-7c35 krb5kdc[24747](info): commencing operation
krb5kdc: starting...
W20260811 05:13:03.939039 31075 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/test-user.keytab.
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.88.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
Entry for principal kudu/127.30.88.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab.
I20260811 05:13:04.074280 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:45695
--webserver_interface=127.30.88.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:04.187170 24760 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:04.187490 24760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.187559 24760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.187636 24760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:04.191629 24760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:04.191705 24760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.191743 24760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.191780 24760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:04.191813 24760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:04.196862 24760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.254.keytab
--principal=kudu/127.30.88.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.254:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.88.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:04.197927 24760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.199010 24760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.207139 24767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.207147 24768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.207147 24770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.207394 24760 server_base.cc:1034] running on GCE node
I20260811 05:13:04.208038 24760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.209133 24760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.210296 24760 hybrid_clock.cc:648] HybridClock initialized: now 1786425184210274 us; error 33 us; skew 500 ppm
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:04.213049 24760 init.cc:377] Logged in from keytab as kudu/127.30.88.254@KRBTEST.COM (short username kudu)
I20260811 05:13:04.213935 24760 webserver.cc:460] Webserver started at http://127.30.88.254:41627/ using document root <none> and password file <none>
I20260811 05:13:04.214444 24760 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.214504 24760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.214668 24760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.216749 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data/instance:
uuid: "ca3077d787d9495d8099e17484037012"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "e608958de9344e2e25272c4a595fb4a5"
server_key_iv: "7628e67076fcb2cce9c855cb1c6bf3d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.217291 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance:
uuid: "ca3077d787d9495d8099e17484037012"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "e608958de9344e2e25272c4a595fb4a5"
server_key_iv: "7628e67076fcb2cce9c855cb1c6bf3d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.221177 24760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 05:13:04.223515 24778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.224725 24760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:04.224861 24760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal
uuid: "ca3077d787d9495d8099e17484037012"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "e608958de9344e2e25272c4a595fb4a5"
server_key_iv: "7628e67076fcb2cce9c855cb1c6bf3d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.224980 24760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.236351 24760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.239117 24760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.239331 24760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.246109 24760 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.254:45695
I20260811 05:13:04.246385 24839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.254:45695 every 8 connection(s)
I20260811 05:13:04.247164 24760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:04.249943 24840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.250914 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24760
I20260811 05:13:04.251057 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-0/wal/instance
I20260811 05:13:04.251399 31075 external_mini_cluster.cc:1428] Setting key cc22bfa7c31e64040f0d066073759e8f
I20260811 05:13:04.254807 24840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.88.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.253@KRBTEST.COM" created.
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:13:04.271130 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.257125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.30.88.254:45695)
0811 05:13:04.257544 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.257562 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.257754 (+ 192us) server_negotiation.cc:407] Connection header received
0811 05:13:04.259503 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.259513 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.259790 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.259985 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.261149 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.262088 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.263164 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.263531 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.263654 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.266527 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.266533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.266542 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.266562 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.268153 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.268664 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.268673 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.268678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.268827 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.269150 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.269155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.269158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.269708 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.269864 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.270230 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.270408 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"spinlock_wait_cycles":27008,"thread_start_us":157,"threads_started":1}
I20260811 05:13:04.271128 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.257126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45695 (local address 127.0.0.1:50852)
0811 05:13:04.257586 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.257638 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.258991 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.260094 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.260118 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.260585 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.260975 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.261002 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.262208 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.262211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.263049 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.263060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.263663 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.263666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.263760 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.264346 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.264391 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.266345 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.268265 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.268275 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.268284 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.268511 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.268943 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.268946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.268947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.269053 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.269835 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.269839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.269958 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.270102 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.270328 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260811 05:13:04.273540 24840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.274197 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.273969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36608)
0811 05:13:04.274080 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.274150 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:13:04.274376 24784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
I20260811 05:13:04.275362 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.275173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36622)
0811 05:13:04.275273 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.275316 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:13:04.275547 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:35747: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
Entry for principal kudu/127.30.88.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab.
I20260811 05:13:04.282147 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:35747
--webserver_interface=127.30.88.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:04.324765 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } attempt: 1
I20260811 05:13:04.325676 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.325383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36638)
0811 05:13:04.325535 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.325601 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 05:13:04.326442 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.326241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36642)
0811 05:13:04.326343 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.326389 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:13:04.326611 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:35747: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
W20260811 05:13:04.392172 24848 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:04.392473 24848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.392562 24848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.392674 24848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:13:04.394761 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } attempt: 2
I20260811 05:13:04.395643 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.395357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36646)
0811 05:13:04.395505 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.395569 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 05:13:04.396344 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.396126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36648)
0811 05:13:04.396238 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.396291 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:13:04.396472 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.253:35747: Network error: Client connection negotiation failed: client connection to 127.30.88.253:35747: connect: Connection refused (error 111)
W20260811 05:13:04.397127 24848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:04.397171 24848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.397189 24848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.397236 24848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:04.397267 24848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:04.402247 24848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.253.keytab
--principal=kudu/127.30.88.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.253:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.88.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:04.403258 24848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.404191 24848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.412395 24856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.412456 24854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.412402 24853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.412554 24848 server_base.cc:1034] running on GCE node
I20260811 05:13:04.413257 24848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.414283 24848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.415439 24848 hybrid_clock.cc:648] HybridClock initialized: now 1786425184415421 us; error 27 us; skew 500 ppm
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:04.418111 24848 init.cc:377] Logged in from keytab as kudu/127.30.88.253@KRBTEST.COM (short username kudu)
I20260811 05:13:04.419121 24848 webserver.cc:460] Webserver started at http://127.30.88.253:43637/ using document root <none> and password file <none>
I20260811 05:13:04.419776 24848 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.419838 24848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.420140 24848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.422606 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data/instance:
uuid: "e85adaa64f2b4883969b81bd120bdf28"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "ddddc890a3ac3504d3a197f73062e423"
server_key_iv: "e64ac8bb1ed60a559db8ca07ea6d3d05"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.423166 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal/instance:
uuid: "e85adaa64f2b4883969b81bd120bdf28"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "ddddc890a3ac3504d3a197f73062e423"
server_key_iv: "e64ac8bb1ed60a559db8ca07ea6d3d05"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.426546 24848 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 05:13:04.429020 24865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.430203 24848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:04.430326 24848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal
uuid: "e85adaa64f2b4883969b81bd120bdf28"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "ddddc890a3ac3504d3a197f73062e423"
server_key_iv: "e64ac8bb1ed60a559db8ca07ea6d3d05"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.430421 24848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.449551 24848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.452317 24848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.452520 24848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.459457 24848 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.253:35747
I20260811 05:13:04.459749 24923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.253:35747 every 8 connection(s)
I20260811 05:13:04.460393 24848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:04.463013 24924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.466372 24924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.468181 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24848
I20260811 05:13:04.468305 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-1/wal/instance
I20260811 05:13:04.468611 31075 external_mini_cluster.cc:1428] Setting key f7f7e2ba89861f2ef98bbddd1a48ce09
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.88.252@KRBTEST.COM; defaulting to no policy
I20260811 05:13:04.480121 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.468097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.30.88.254:45695)
0811 05:13:04.468253 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.468259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.469192 (+ 933us) server_negotiation.cc:407] Connection header received
0811 05:13:04.470100 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.470103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.470173 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.470260 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.471806 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.472348 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.473472 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.473722 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.473833 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.476601 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.476606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.476610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.476645 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.478215 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.478737 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.478739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.478740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.478818 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.479189 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.479191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.479193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.479356 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.479431 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.479899 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.480004 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.88.252@KRBTEST.COM" created.
I20260811 05:13:04.481091 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.468554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45695 (local address 127.0.0.1:50866)
0811 05:13:04.469046 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.469086 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.469875 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.470451 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.470461 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.471015 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.471655 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.471671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.472493 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.472496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.473346 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.473356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.473871 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.473875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.474056 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.474665 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.474681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.476421 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.478334 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.478339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.478349 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.478599 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.478972 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.478974 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.478976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.479105 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.479445 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.479449 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.479668 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.479853 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.480119 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":131,"threads_started":1}
I20260811 05:13:04.482527 24924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.487651 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } attempt: 3
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:04.492688 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.483067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36654)
0811 05:13:04.483173 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.483187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.483265 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.483864 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.483866 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.484016 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.484103 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.484108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.485599 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.485601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.486159 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.486167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.487039 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.487042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.487171 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.487582 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.487591 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.488968 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.491144 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.491152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.491343 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.491673 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.491677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.491758 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.492194 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.492197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.492279 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.492390 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.492527 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":34}
I20260811 05:13:04.492985 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.483142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.30.88.253:35747)
0811 05:13:04.483456 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.483470 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.483492 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:13:04.483539 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.483547 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.483732 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.483871 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.484225 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.485472 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.486279 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.486924 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.487124 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.489098 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.489109 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.489124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.489142 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.491042 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491453 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.491459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.491461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.491568 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491845 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.491850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.491852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.492080 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.492218 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.492477 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.492593 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":213,"thread_start_us":125,"threads_started":1}
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:04.494671 24924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.495385 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.495111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56610)
0811 05:13:04.495221 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.495307 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 05:13:04.495604 24868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
I20260811 05:13:04.496439 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.496217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56626)
0811 05:13:04.496325 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.496372 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:13:04.496651 24924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
Entry for principal kudu/127.30.88.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab.
I20260811 05:13:04.497467 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.488028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36664)
0811 05:13:04.488155 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.488168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.488238 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.488805 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.488810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.489018 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.489128 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.489138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.490099 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.490104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.491276 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.491286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.491941 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.491944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.492047 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.492551 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.492565 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.493943 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.495912 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.495915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.495917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.496162 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.496453 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.496456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.496457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.496594 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.496966 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.496969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.497047 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.497171 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.497294 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:13:04.497624 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.488250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.30.88.253:35747)
0811 05:13:04.488507 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.488515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.488528 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:04.488558 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.488560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.488617 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.488775 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.489267 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.489970 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.491400 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.491830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.491939 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.494095 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.494098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.494101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.494125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.495808 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.496278 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.496283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.496284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.496367 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.496682 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.496685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.496686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.496897 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.496975 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.497229 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.497341 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260811 05:13:04.498257 31075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
/tmp/dist-test-taskwl2kZi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:37669
--webserver_interface=127.30.88.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.88.212:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:04.498282 24840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.498919 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.498666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56640)
0811 05:13:04.498773 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.498818 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:13:04.499615 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.499341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56654)
0811 05:13:04.499483 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.499538 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 05:13:04.499758 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
I20260811 05:13:04.530920 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 1
I20260811 05:13:04.531821 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.531469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56662)
0811 05:13:04.531644 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.531720 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 05:13:04.532609 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.532363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56670)
0811 05:13:04.532491 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.532541 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:13:04.532757 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
I20260811 05:13:04.545922 24924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 1
I20260811 05:13:04.546624 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.546382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56674)
0811 05:13:04.546511 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.546562 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:13:04.547331 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.547117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56682)
0811 05:13:04.547224 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.547274 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:13:04.547456 24924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
W20260811 05:13:04.606487 24932 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:13:04.606770 24932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.606842 24932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.606941 24932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:13:04.607928 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 2
I20260811 05:13:04.608803 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.608462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56688)
0811 05:13:04.608644 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.608712 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:13:04.609535 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.609302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56694)
0811 05:13:04.609423 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.609472 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 05:13:04.609668 24840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
W20260811 05:13:04.611452 24932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:04.611505 24932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.611548 24932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.611599 24932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:04.611634 24932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:04.615621 24924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 2
I20260811 05:13:04.616261 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.616012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56704)
0811 05:13:04.616131 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.616192 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 05:13:04.616956 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.616750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56710)
0811 05:13:04.616853 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.616899 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:13:04.616925 24932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.88.212:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.88.254:45695,127.30.88.253:35747,127.30.88.252:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/krb5kdc/kudu_127.30.88.252.keytab
--principal=kudu/127.30.88.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.88.252:37669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.88.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
W20260811 05:13:04.617249 24924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.88.252:37669: Network error: Client connection negotiation failed: client connection to 127.30.88.252:37669: connect: Connection refused (error 111)
I20260811 05:13:04.617921 24932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.618923 24932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.627048 24938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.627051 24941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.627106 24939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.627370 24932 server_base.cc:1034] running on GCE node
I20260811 05:13:04.627931 24932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.629173 24932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.630379 24932 hybrid_clock.cc:648] HybridClock initialized: now 1786425184630356 us; error 55 us; skew 500 ppm
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:04.633040 24932 init.cc:377] Logged in from keytab as kudu/127.30.88.252@KRBTEST.COM (short username kudu)
I20260811 05:13:04.634020 24932 webserver.cc:460] Webserver started at http://127.30.88.252:42197/ using document root <none> and password file <none>
I20260811 05:13:04.634614 24932 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.634721 24932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.635023 24932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.637030 24932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data/instance:
uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "0f47fce11ad9edca34f957ec4a32c0c4"
server_key_iv: "35a816e916341969114e446d302dc46f"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.637650 24932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal/instance:
uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "0f47fce11ad9edca34f957ec4a32c0c4"
server_key_iv: "35a816e916341969114e446d302dc46f"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.641307 24932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.005s
I20260811 05:13:04.643654 24949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.644853 24932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:04.645001 24932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data,/tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal
uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-7c35"
server_key: "0f47fce11ad9edca34f957ec4a32c0c4"
server_key_iv: "35a816e916341969114e446d302dc46f"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.645108 24932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.664465 24932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.667178 24932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.667368 24932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.674026 24932 rpc_server.cc:307] RPC server started. Bound to: 127.30.88.252:37669
I20260811 05:13:04.674279 25010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.88.252:37669 every 8 connection(s)
I20260811 05:13:04.675024 24932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:04.677618 25011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.681037 25011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.684167 31075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu as pid 24932
I20260811 05:13:04.684291 31075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwl2kZi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786424902138073-31075-0/minicluster-data/master-2/wal/instance
I20260811 05:13:04.684604 31075 external_mini_cluster.cc:1428] Setting key 256dd6cb30f3c7e01ed37dc66018eaee
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:13:04.695736 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.682764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.30.88.254:45695)
0811 05:13:04.682927 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.682931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.683688 (+ 757us) server_negotiation.cc:407] Connection header received
0811 05:13:04.684776 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.684779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.684839 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.684940 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.686417 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.687148 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.688561 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.688908 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.689024 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.691805 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.691809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.691812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.691839 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.693528 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.694030 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.694036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.694040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.694134 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.694494 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.694497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.694498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.694683 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.694804 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.695418 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.695561 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:04.696463 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.685271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.30.88.252:37669)
0811 05:13:04.685676 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.685686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.685710 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:13:04.685772 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.685777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.685942 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.686107 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.686961 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.688231 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.689299 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.689788 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.689905 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.692607 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.692620 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.692630 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.692668 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.694337 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.694763 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.694771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.694776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.694897 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.695191 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.695193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.695194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.695461 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.695683 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.696153 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.696282 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260811 05:13:04.696467 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.683137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.254:45695 (local address 127.0.0.1:50880)
0811 05:13:04.683549 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.683586 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.684527 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.685138 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.685147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.685658 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.686263 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.686277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.687253 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.687257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.688446 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.688453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.689039 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.689042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.689188 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.689716 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.689736 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.691669 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.693632 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.693640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.693648 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.693925 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.694234 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.694236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.694238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.694359 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.694814 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.694819 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.695161 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.695339 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.695609 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":46,"thread_start_us":99,"threads_started":1}
I20260811 05:13:04.697655 25011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.254@KRBTEST.COM
I20260811 05:13:04.705722 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.698106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.253:35747 (local address 127.0.0.1:36666)
0811 05:13:04.698216 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.698227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.698311 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.698600 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.698603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.698763 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.698899 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.698910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.699621 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.699627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.700113 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.700121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.700597 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.700600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.700696 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.701133 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.701152 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.702291 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.704156 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.704159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.704161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.704463 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.704787 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.704791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.704796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.704895 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.705250 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.705253 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.705354 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.705509 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.705630 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:13:04.705886 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.698245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.30.88.253:35747)
0811 05:13:04.698371 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.698375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.698389 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:04.698433 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.698436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.698489 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.698586 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.699022 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.699516 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.700230 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.700482 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.700585 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.702425 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.702428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.702431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.702463 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.704018 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.704591 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.704595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.704598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.704691 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.704991 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.704994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.704995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.705149 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.705222 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.705596 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.705716 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:13:04.706606 25011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:04.707448 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.698768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.30.88.254:45695)
0811 05:13:04.698892 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.698896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.698933 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:04.699064 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.699067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.699112 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.699187 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.699714 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.700161 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.700862 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.701077 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.701181 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.703764 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.703768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.703770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.703798 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.705437 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.705921 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.705926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.705928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.706031 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.706409 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.706413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.706414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.706665 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.706920 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.707189 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.707296 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:13:04.708428 24924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 3
I20260811 05:13:04.708788 24840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } attempt: 3
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.252@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.253@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.88.254@KRBTEST.COM for kudu/127.30.88.252@KRBTEST.COM
I20260811 05:13:04.715737 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.707079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56726)
0811 05:13:04.707236 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.707249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.707343 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.707573 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.707576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.707709 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.707800 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.707807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.708679 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.708685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.709634 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.709643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.710283 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.710289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.710359 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.710809 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.710822 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.712172 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.714297 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.714300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.714302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.714485 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.714759 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.714762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.714764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.714885 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.715283 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.715288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.715376 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.715468 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.715594 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"spinlock_wait_cycles":1664}
I20260811 05:13:04.715891 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.707152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.30.88.252:37669)
0811 05:13:04.707238 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.707243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.707293 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:13:04.707441 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.707444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.707478 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.707552 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.707923 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.708585 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.709770 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.710168 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.710554 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.712292 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.712295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.712297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.712316 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.714172 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.714589 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.714594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.714596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.714671 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.714983 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.714986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.714987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.715181 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.715580 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.715643 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.715726 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":28}
Aug 11 05:13:04 dist-test-slave-7c35 krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.88.253@KRBTEST.COM
I20260811 05:13:04.719441 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.709108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56748)
0811 05:13:04.709246 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.709259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.709321 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.710119 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.710122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.710309 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.710393 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.710399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.711714 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.711718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.712525 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.712539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.713280 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.713284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.713364 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.714009 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.714030 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.715653 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.717910 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.717916 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.717918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.718193 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.718465 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.718469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.718554 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.719004 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.719007 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.719102 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.719180 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.719287 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 05:13:04.719852 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.708745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.88.252:37669 (local address 127.0.0.1:56736)
0811 05:13:04.708890 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.708904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.708966 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.709486 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.709492 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.709691 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.709777 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.709783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.711445 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.711448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.712203 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.712211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.712937 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.712940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.713015 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.713478 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.713499 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.715201 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.718244 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.718250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.718481 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.718786 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.718791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.718905 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.719381 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.719387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.719470 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.719552 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.719698 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 05:13:04.720098 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.709476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.30.88.252:37669)
0811 05:13:04.709881 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.709888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.709897 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:04.709923 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.709925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.709963 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.710510 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.710695 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.711573 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.712682 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.713127 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.713277 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.715802 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.715805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.715809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.715834 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.717784 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718290 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.718295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.718296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.718374 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718656 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.718659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.718661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.718878 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.719716 (+ 838us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.719802 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.719904 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":351,"thread_start_us":101,"threads_started":1}
I20260811 05:13:04.720350 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.708551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.30.88.253:35747)
0811 05:13:04.708644 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.708649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.708820 (+ 171us) server_negotiation.cc:407] Connection header received
0811 05:13:04.708996 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.709000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.709042 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.709090 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.709910 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.711038 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.712092 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.712402 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.712464 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.716656 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.716659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.716661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.716683 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.718244 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718929 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.718935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.718938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.719019 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.719336 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.719341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.719344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.719540 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.719706 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.720086 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.720194 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37,"spinlock_wait_cycles":1792}
I20260811 05:13:04.720497 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.708778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.30.88.252:37669)
0811 05:13:04.709236 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.709245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.709256 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:04.709291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.709295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.709332 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.709892 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.710007 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.711325 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.712373 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.712813 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.713212 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.715380 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.715384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.715386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.715405 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.718102 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.718587 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.718593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.718596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.718674 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.719046 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.719051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.719054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.719257 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.720138 (+ 881us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.720218 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.720318 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":396,"spinlock_wait_cycles":2944,"thread_start_us":92,"threads_started":1}
I20260811 05:13:04.721071 25011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: Bootstrap starting.
I20260811 05:13:04.722656 24924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: Bootstrap starting.
I20260811 05:13:04.722757 24840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: Bootstrap starting.
I20260811 05:13:04.723794 25011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.724650 25011 log.cc:824] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.725033 24924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.725169 24840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.725795 24924 log.cc:824] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.725925 24840 log.cc:824] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.726879 25011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: No bootstrap required, opened a new log
I20260811 05:13:04.727674 24924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: No bootstrap required, opened a new log
I20260811 05:13:04.728446 24840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: No bootstrap required, opened a new log
I20260811 05:13:04.728993 31075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:04.730273 24809 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ca3077d787d9495d8099e17484037012"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50886
I20260811 05:13:04.731511 24924 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.731516 25011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.731737 24924 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85adaa64f2b4883969b81bd120bdf28, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.731729 25011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf9cd3d89734de0bb9cbce8a1b3331d, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.732065 25011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.732065 24924 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.732545 24840 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.732761 24840 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3077d787d9495d8099e17484037012, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.733156 24840 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.733014 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.733464 25011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:04.733629 24924 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:04.732960 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.733904 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.733822 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.734503 24840 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:04.734596 24809 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:13:04.734697 24809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.736047 24809 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.737450 24895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca3077d787d9495d8099e17484037012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e85adaa64f2b4883969b81bd120bdf28"
I20260811 05:13:04.737696 24895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.739111 24895 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca3077d787d9495d8099e17484037012 in term 1.
I20260811 05:13:04.737001 24809 leader_election.cc:288] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [CANDIDATE]: Term 1 election: Requested vote from peers e85adaa64f2b4883969b81bd120bdf28 (127.30.88.253:35747), 0cf9cd3d89734de0bb9cbce8a1b3331d (127.30.88.252:37669)
I20260811 05:13:04.734517 25026 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.740460 25026 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:13:04.740868 24787 tablet.cc:2369] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:04.741103 25060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:04.741217 25060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 05:13:04.741931 25054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:04.742007 25054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:04.737808 24983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca3077d787d9495d8099e17484037012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d"
W20260811 05:13:04.739279 25053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 05:13:04.743098 24983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.743456 24784 leader_election.cc:302] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca3077d787d9495d8099e17484037012, e85adaa64f2b4883969b81bd120bdf28; no voters:
I20260811 05:13:04.743844 25026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:04.744068 25026 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [term 1 LEADER]: Becoming Leader. State: Replica: ca3077d787d9495d8099e17484037012, State: Running, Role: LEADER
I20260811 05:13:04.744380 25026 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } }
I20260811 05:13:04.744524 24983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca3077d787d9495d8099e17484037012 in term 1.
W20260811 05:13:04.744808 25053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:04.746703 25071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca3077d787d9495d8099e17484037012. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.746902 25071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:04.748207 24848 master_runner.cc:263] No leader master found from master e85adaa64f2b4883969b81bd120bdf28
I20260811 05:13:04.748275 24848 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:13:04.750494 24760 master_runner.cc:263] No leader master found from master ca3077d787d9495d8099e17484037012
I20260811 05:13:04.750557 24760 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:13:04.752270 25075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:04.754832 25075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:04.758100 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.742914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.30.88.253:35747)
0811 05:13:04.743085 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.743090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.746229 (+ 3139us) server_negotiation.cc:407] Connection header received
0811 05:13:04.746426 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.746430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.746493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.746553 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.748104 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.748881 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.752042 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.752382 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.752485 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.754310 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.754313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.754316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.754340 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.756154 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.756683 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.756689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.756691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.756783 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.757104 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.757108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.757111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.757333 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.757445 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.757828 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.757949 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 05:13:04.760697 24983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [term 1 FOLLOWER]: Refusing update from remote peer ca3077d787d9495d8099e17484037012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:04.761353 25071 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:04.765228 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca3077d787d9495d8099e17484037012. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.765338 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.766237 24895 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [term 1 FOLLOWER]: Refusing update from remote peer ca3077d787d9495d8099e17484037012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:04.766788 25071 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:04.768790 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.744015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.30.88.252:37669)
0811 05:13:04.744591 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.744596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.758978 (+ 14382us) server_negotiation.cc:407] Connection header received
0811 05:13:04.759829 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.759832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.759897 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.759955 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.760856 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.761628 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.762757 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.763135 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.763217 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.764937 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.764941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.764944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.764969 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.766797 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.767376 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.767380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.767382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.767470 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.767797 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.767801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.767803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.768017 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.768118 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.768506 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.768629 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473}
I20260811 05:13:04.770046 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca3077d787d9495d8099e17484037012. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.770176 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.772480 25026 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.772616 25026 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:04.772852 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.772954 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.773404 25076 mvcc.cc:205] Tried to move back new op lower bound from 7317197556770168832 to 7317197556716654592. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556770168832}]
I20260811 05:13:04.774111 25075 catalog_manager.cc:1347] Generated new cluster ID: bc7517d001244b4b9a8ab34f5a08d60b
I20260811 05:13:04.774174 25075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:04.781256 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.743321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.30.88.254:45695)
0811 05:13:04.744784 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.744788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.758712 (+ 13924us) server_negotiation.cc:407] Connection header received
0811 05:13:04.759586 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.759589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.759637 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.759687 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.760741 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.761525 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.775109 (+ 13584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.775456 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.775563 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.777325 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.777330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.777334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.777365 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.779058 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.779543 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.779547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.779549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.779632 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.779931 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.779934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.779935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.780157 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.780279 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.780958 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.781091 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1390}
I20260811 05:13:04.783838 25080 mvcc.cc:205] Tried to move back new op lower bound from 7317197556770168832 to 7317197556716654592. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556770168832}]
I20260811 05:13:04.784461 25075 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:04.785607 25075 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:04.792826 25071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.792955 25071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:04.794330 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca3077d787d9495d8099e17484037012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3077d787d9495d8099e17484037012" member_type: VOTER last_known_addr { host: "127.30.88.254" port: 45695 } } peers { permanent_uuid: "e85adaa64f2b4883969b81bd120bdf28" member_type: VOTER last_known_addr { host: "127.30.88.253" port: 35747 } } peers { permanent_uuid: "0cf9cd3d89734de0bb9cbce8a1b3331d" member_type: VOTER last_known_addr { host: "127.30.88.252" port: 37669 } } }
I20260811 05:13:04.794454 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:04.800705 25078 mvcc.cc:205] Tried to move back new op lower bound from 7317197556770168832 to 7317197556716654592. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556770168832}]
I20260811 05:13:04.808475 25075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca3077d787d9495d8099e17484037012: Generated new TSK 0
I20260811 05:13:04.809284 25075 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:13:04.846758 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.843453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.30.88.253:35747)
0811 05:13:04.843585 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.843590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.845564 (+ 1974us) server_negotiation.cc:407] Connection header received
0811 05:13:04.845818 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.845821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.845888 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.845943 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.846617 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36684: BlockingRecv error: recv got EOF from 127.0.0.1:36684 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 05:13:04.848248 25018 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.844275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.30.88.252:37669)
0811 05:13:04.844390 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.844394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.847293 (+ 2899us) server_negotiation.cc:407] Connection header received
0811 05:13:04.847467 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.847471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.847530 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.847581 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.848132 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56770: BlockingRecv error: recv got EOF from 127.0.0.1:56770 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 05:13:04.849723 24845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.844955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.30.88.254:45695)
0811 05:13:04.845070 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.845074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.848793 (+ 3719us) server_negotiation.cc:407] Connection header received
0811 05:13:04.848975 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.848978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.849030 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.849087 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.849609 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50912: BlockingRecv error: recv got EOF from 127.0.0.1:50912 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 05:13:04.853955 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.852542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.30.88.253:35747)
0811 05:13:04.852631 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.852635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.853031 (+ 396us) server_negotiation.cc:407] Connection header received
0811 05:13:04.853243 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.853247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.853321 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.853372 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.853863 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36688: BlockingRecv error: recv got EOF from 127.0.0.1:36688 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 05:13:04.859745 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.857291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.30.88.253:35747)
0811 05:13:04.857377 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.857381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.857832 (+ 451us) server_negotiation.cc:407] Connection header received
0811 05:13:04.859129 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.859132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.859178 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.859224 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.859665 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36698: BlockingRecv error: recv got EOF from 127.0.0.1:36698 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 05:13:04.865097 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.863719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.30.88.253:35747)
0811 05:13:04.863805 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.863808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.864286 (+ 478us) server_negotiation.cc:407] Connection header received
0811 05:13:04.864455 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.864458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.864501 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.864548 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.865004 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36710: BlockingRecv error: recv got EOF from 127.0.0.1:36710 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 05:13:04.872485 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.871063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.30.88.253:35747)
0811 05:13:04.871181 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.871185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.871641 (+ 456us) server_negotiation.cc:407] Connection header received
0811 05:13:04.871816 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.871819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.871887 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.871939 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.872404 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36724: BlockingRecv error: recv got EOF from 127.0.0.1:36724 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 05:13:04.884153 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.882571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.30.88.253:35747)
0811 05:13:04.882673 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.882677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.883254 (+ 577us) server_negotiation.cc:407] Connection header received
0811 05:13:04.883447 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.883450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.883498 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.883548 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.884052 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36740: BlockingRecv error: recv got EOF from 127.0.0.1:36740 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:13:04.904222 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.902366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.30.88.253:35747)
0811 05:13:04.902484 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.902489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.903149 (+ 660us) server_negotiation.cc:407] Connection header received
0811 05:13:04.903367 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.903370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.903426 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.903485 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.904105 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36744: BlockingRecv error: recv got EOF from 127.0.0.1:36744 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 05:13:04.939944 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:04.938564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.30.88.253:35747)
0811 05:13:04.938692 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.938697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.939006 (+ 309us) server_negotiation.cc:407] Connection header received
0811 05:13:04.939178 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.939182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.939243 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.939333 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.939801 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36760: BlockingRecv error: recv got EOF from 127.0.0.1:36760 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 05:13:05.007684 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:05.006265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.30.88.253:35747)
0811 05:13:05.006435 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.006440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.006663 (+ 223us) server_negotiation.cc:407] Connection header received
0811 05:13:05.006920 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.006924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.006984 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.007079 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.007568 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36764: BlockingRecv error: recv got EOF from 127.0.0.1:36764 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 05:13:05.139166 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:05.138051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.30.88.253:35747)
0811 05:13:05.138212 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.138216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.138266 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:13:05.138420 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.138422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.138484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.138558 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.139041 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36772: BlockingRecv error: recv got EOF from 127.0.0.1:36772 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 05:13:05.398919 24930 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:05.397716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.30.88.253:35747)
0811 05:13:05.397871 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.397875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.397938 (+ 63us) server_negotiation.cc:407] Connection header received
0811 05:13:05.398084 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.398086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.398132 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.398202 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.398746 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36788: BlockingRecv error: recv got EOF from 127.0.0.1:36788 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:13:05.743402 25054 catalog_manager.cc:1259] Loaded cluster ID: bc7517d001244b4b9a8ab34f5a08d60b
I20260811 05:13:05.743491 25054 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: loading cluster ID for follower catalog manager: success
I20260811 05:13:05.745170 25054 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: acquiring CA information for follower catalog manager: success
I20260811 05:13:05.745929 25053 catalog_manager.cc:1259] Loaded cluster ID: bc7517d001244b4b9a8ab34f5a08d60b
I20260811 05:13:05.745993 25053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: loading cluster ID for follower catalog manager: success
I20260811 05:13:05.746382 25054 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e85adaa64f2b4883969b81bd120bdf28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:05.747478 25053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: acquiring CA information for follower catalog manager: success
I20260811 05:13:05.748498 25053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cf9cd3d89734de0bb9cbce8a1b3331d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:05.916803 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.913090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.30.88.253:35747)
0811 05:13:05.913288 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.913292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.913347 (+ 55us) server_negotiation.cc:407] Connection header received
0811 05:13:05.913476 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.913478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.913711 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.913821 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:05.914055 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.914590 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.915421 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.915652 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.915727 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.915977 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:05.916393 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:05.916496 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.916541 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.916586 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260811 05:13:05.923132 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.919674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.30.88.252:37669)
0811 05:13:05.919919 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.919926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.919942 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:13:05.920026 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.920029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.920146 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.920266 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:05.920481 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.920985 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.921781 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.922009 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.922118 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.922411 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:05.922749 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:05.922901 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.922938 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.922989 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":55,"threads_started":1}
I20260811 05:13:05.929945 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.926253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.30.88.252:37669)
0811 05:13:05.926349 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.926355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.926466 (+ 111us) server_negotiation.cc:407] Connection header received
0811 05:13:05.926642 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.926645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.926706 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.926767 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:05.927141 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.927858 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.928849 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.929142 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.929242 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.929483 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:05.929549 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:05.929634 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.929740 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.929792 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:13:05.930402 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.925890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.30.88.253:35747)
0811 05:13:05.925998 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.926002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.926410 (+ 408us) server_negotiation.cc:407] Connection header received
0811 05:13:05.926559 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.926562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.926608 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.926673 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:05.927000 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.927758 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.928885 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.929209 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.929304 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.929639 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:05.929735 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:05.929793 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.930216 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.930263 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:13:05.930629 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.925843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.30.88.254:45695)
0811 05:13:05.926145 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.926151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.926169 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:13:05.926265 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.926268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.926422 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.926579 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:05.926804 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.927462 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.928602 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.928958 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.929057 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.929576 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:05.929996 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:05.930375 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.930412 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.930480 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":58,"threads_started":1}
I20260811 05:13:05.937944 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24760
I20260811 05:13:05.950059 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24848
I20260811 05:13:05.958213 31075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwl2kZi/build/debug/bin/kudu with pid 24932
2026-08-11T05:13:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3132 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6436 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283814 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST